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ABSTRACT 



The National Assessment of Educational Progress (NAEP) is 
the nation's only ongoing representative sample survey of student achievement 
in core subject areas. In 2001, NAEP conducted a national United States 
history assessment of fourth-, eighth-, and twelfth-grade students. This 
report presents a summary of the results of the NAEP 2001 U.S. history 
assessment for the nation. Results in 2001 are compared to results in 1994, 
the next most recent year in which the NAEP conducted a U.S. history 
assessment and the only other assessment year in which the test questions 
were based on the current framework. Students’ performance on the assessment 
is described in terms of average scores on a 0-500 scale and in terms of the 
percentage of students attaining 3 achievement levels: (1) Basic; (2) 

Proficient; and (3) Advanced. The report provides results for subgroups of 
students at the three grade levels defined by various background and 
contextual characteristics . Appended are an overview of the procedures used 
in the NAEP assessment and data collection. (BT) 
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What 5$ The Nation’s IRepoct Card? 

THE NATION’S REPORT CARD, the National Assessment of Educational Progress (NAEP),is the only nationally 
representative and continuing assessment of what America’s students know and can do in various subject areas. Since 
1969, assessments have been conducted periodically in reading, mathematics, science, writing, history, geography, and 
other fields. By making objective information on student performance available to policymakers at the national, state, 
and local levels, NAEP is an integral part of our nation’s evaluation of the condition and progress of education. Only 
information related to academic achievement is collected under this program. NAEP guarantees the privacy of 
individual students and their families. 

NAEP is a congressionally mandated project of the National Center for Education Statistics, the U.S. Department 
of Education. The Commissioner of Education Statistics is responsible, by law, for carrying out the NAEP project 
through competitive awards to qualified organizations. NAEP reports directly to the Commissioner, who is also 
responsible for providing continuing reviews, including validation studies and solicitation of public comment, on 
NAEP’s conduct and usefulness. 

In 1988, Congress established the National Assessment Governing Board (NAGB) to formulate policy guidelines 
for NAEP. The Board is responsible for selecting the subject areas to be assessed from among those included in the 
National Education Goals; for setting appropriate student performance levels; for developing assessment objectives and 
test specifications through a national consensus approach; for designing the assessment methodology; for developing 
guidelines for reporting and disseminating NAEP results; for developing standards and procedures for interstate, 
regional, and national comparisons; for determining the appropriateness of test items and ensuring they are free from 
bias; and for taking actions to improve the form and use of the National Assessment. 
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ixecutive Summary 



The National Assessment of Educational Progress (NAEP) is 
the nation’s only ongoing representative sample survey of 
student achievement in core subject areas. In 2001, NAEP 
conducted a national U.S. history assessment of fourth-, 
eighth-, and twelfth-grade students. 

Authorized by Congress and administered by the National 
Center for Education Statistics (NCES) in the U.S. 
Department of Education, NAEP regularly reports to the 
public on the educational progress of students in grades 4, 8, 
and 12. This report presents the results of the NAEP 2001 
U.S. history assessment for the nation. Results in 2001 are 
compared to results in 1994, the next most recent year in 
which NAEP conducted a U.S. history assessment and the 
only other assessment year in which the test questions were 
based on the current framework. Students’ performance on 
the assessment is described in terms of average scores on a 
0-500 scale and in terms of the percentage of students 
attaining three achievement levels: Basic, Proficient, and 
Advanced. The achievement levels are performance standards 
adopted by the National Assessment Governing Board 
(NAGB) as part of its statutory responsibilities. They are 
collective judgments of what students should know and be 
able to do. 
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As provided by law, the Deputy Com- 
missioner of Education Statistics, upon 
review of a congressionally mandated 
evaluation ofNAEP, has determined that 
the achievement levels are to be used on a 
trial basis and should be interpreted and 
used with caution. However, both the 
Deputy Commissioner and NAGB believe 
these performance standards are useful for 
understanding trends in student achieve- 
ment. They have been widely used by 
national and state officials as a common 
yardstick of academic performance. 

In addition to providing average scores 
and achievement-level performance in U.S. 
history for the nation s fourth-, eighth-, 
and twelfth-graders, this report provides 
results for subgroups of students at those 
grade levels defined by various background 
and contextual characteristics. 

A summary of major findings from the 
NAEP 2001 U.S. history assessment is 
presented on the following pages. In 
interpreting NAEP results, it should be 
noted that every test score has a standard 
error — a range of a few points plus or 
minus the score — that includes 
components of sampling error and 
measurement error. Statistical tests that 
factor in these standard errors are used to 
determine whether the differences between 
average scores are significant. Only 
statistically significant differences are cited 
in this report. Readers are also cautioned 
against making causal inferences based on 
NAEP results. Differences in performance 
between subgroups of students, for 
example, reflect a variety of socioeconomic 
and educational factors. 



Major Findings at 
Grades 4, 8, and 12 

□ Average U.S. history scores for fourth- 
and eighth-graders were higher in 
2001 than in 1994, while the perfor- 
mance of twelfth-graders remained 
relatively stable. 

□ Score increases were evident among the 
lower-performing students at grade 4 (at 
the 10th and 25th percentiles) and for 
both lower- and higher-performing 
students at grade 8 (25th, 75th, and 90th 
percentiles). 

□ Results of the 2001 U.S. history assess- 
ment show 18 percent of fourth-graders, 
17 percent of eighth-graders, and 11 
percent of twelfth-graders performing at 
or above the Proficient level — identified 
by NAGB as the level at which all 
students should perform. 

□ At grade 4, the percentage of students 
performing at or above Basic in 2001 was 
higher than in 1994. At grade 8, the 
percentages of students performing at or 
above Basic, at or above Proficient , and at 
Advanced increased between 1994 and 
2001. At grade 12, however, the per- 
centages performing at or above each 
level remained the same as in 1994. 

Results for Student Subgroups 

In addition to overall results, NAEP reports 
on the performance of various subgroups 
of students. Observed differences between 
student subgroups in NAEP U.S. history 
performance reflect a range of socioeco- 
nomic and educational factors not 
addressed in this report or by NAEP 
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Gender 

□ Any apparent differences in the average 
scores of male and female students in 
2001 were not statistically significant at 
any of the three grades. 

□ At grade 4, both male and female stu- 
dents had higher average scores in 2001 
than in 1994. At grade 8, the average 
score of males increased between 1994 
and 2001, while the performance of 
females remained stable. 

Race/Ethnicity 

D In 2001, the average scores of White 
students were higher than those of Black, 
Hispanic, and American Indian students 
at all three grades. Asian/Pacific Islander 
students scored higher than Black 
and Hispanic students across the grades 
as well. 

□ At grade 4, both White students and 
Black students had higher average scores 
in 2001 than in 1994. At grade 8, only 
White students showed a gain since 
1994. At grade 12, only Hispanic stu- 
dents had higher average scores in 2001 
than in 1994. 

□ The 2001 results show a narrowing of 
the score point difference between 
White students and Black students at 
grade 4, and between White students and 
Hispanic students at grade 12. 

Region of the Country 

□ Fourth- and eighth-grade students in the 
Northeast, Southeast, and Central re- 
gions all had higher average scores than 
students in the West. Fourth- and 
eighth-grade students in the Central 
region outperformed their peers in the 
Southeast. There was no statistically 
significant difference in the performance 
of twelfth-graders from various regions 
of the country. 



0 




□ At grade 4, only the Northeast region 
showed a gain in the U.S. history average 
score since 1994. At grade 8, the only 
increase occurred in the Southeast 
region. 

Parents’ Highest Level of Education 

□ The 2001 results show a clear positive 
relationship overall between parental 
education level and the performance of 
eighth- and twelfth-graders. 

□ At grade 8, the average score of students 
whose parents graduated from college 
was higher in 2001 than in 1994. At 
grade 12, there was an increase in the 
average score of students whose parents 
did not finish high school. 

Type of School 

□ The 2001 results show that public school 
students at all three grades had lower 
average U.S. history scores than their 
peers attending nonpublic schools. 

□ Average scores among both fourth- and 
eighth-grade public school students were 
higher in 2001 than in 1994. 

Type of Location 

□ At grades 4 and 8, students attending 
schools in rural and urban fringe loca- 
tions had higher average scores than 
students in central city schools. At grade 
12, students attending schools in urban 
fringe locations had higher scores than 
students in both rural and central city 
locations. 

Eligibility for Free/Reduced-Price 
School Lunch Program 

□ At every grade, the average score of 
students who were eligible for the Free/ 
Reduced-Price School Lunch program 
was lower than the average score of 
students who were not eligible for the 
program (i.e., those not meeting the 
poverty guidelines). 
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Becoming a More inclusive SSSAEP 

In the 2001 U.S. history assessment, the 
NAEP program used a split-sample design, so 
that trends in students’ history achievement 
could be reported across assessment years 
and, at the same time, the program could 
continue to examine the effects of includ- 
ing special-needs students assessed with 
accommodations. While most of the results 
in this report include only the performance 
of students assessed without accommoda- 
tions, the report also presents an overview 
of a second set of results that include the 
performance of special-needs students who 
required and were provided accommoda- 
tions during the assessment administration. 

□ At grade 8, the average score when 
accommodations were permitted was 
lower than the average score when 
accommodations were not permitted. 
However, there were no statistically 
significant differences between average 
scores in the accommodations-permitted 
results and the accommodations-not- 
permitted results at grades 4 and 12. 

Classroom Contexts for Learning 

NAEP collects information about the 
contexts for student learning by adminis- 
tering questionnaires to assessed students, 
their teachers, and their school administra- 
tors. Using the student as the unit of 
analysis, NAEP examines the relationship 
between selected contextual variables drawn 
from these questionnaires and students’ 
average scores on the U. S. history assessment. 

lime Spent on Social Studies 

□ In 2001, fourth-graders whose teachers 
reported spending more than 180 
minutes on social studies instruction in a 
typical week had higher average scores 
than those whose teachers reported 
spending less time. 




State and Local Standards 

□ About two-thirds of the fourth- and 
eighth-graders assessed had teachers who 
reported that they used state or local 
standards to a large extent in planning 
social studies instruction. There were no 
statistically significant differences in 
students’ performance at either grade 4 
or grade 8 based on the extent to which 
teachers reported using such standards 

in planning instruction. 

Instructional Activities 

□ A large majority of fourth-graders had 
teachers who reported having them read 
material from a textbook on a daily or 
weekly basis. Reading from a textbook 
daily was associated with higher average 
scores than was doing so on a weekly or 
monthly basis. 

□ Eighth-graders whose teachers reported 
using primary historical documents such 
as letters, diaries, or essays written by 
historical figures, on a weekly basis had 
higher average scores than those whose 
teachers did so less frequently. 

□ Twelfth-graders who reported never 
reading extra material, such as biogra- 
phies or historical stories, scored lower, 
on average, than those who reported 
doing so a few times a year or more 
often. 

Use of Technology 

□ A strong positive association was evident 
between using computers for conduct- 
ing research and for writing reports and 
performance at grades 8 and 12. 

□ Students in grades 4, 8, and 12 who 
reported daily general use of computers 
at school for social studies or history had 
lower average scores than those who 
reported less frequent general use. It 
should be noted that relatively few 
students reported using a computer for 
history or social studies. 
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NAEP 2001 U.S. History Assessment 



Introduction 

Knowledge of United States history is an important 
component of effective citizenship. Having a thorough grasp 
of our country’s struggles and achievements better enables 
young people to make informed and intelligent decisions 
about contemporary issues. Nourishing the curiosity 
children exhibit about major events, customs and 

institutions, and the families and individuals that 
comprise United States history, creates a valuable 
resource for our nation’s future. 

Efforts to improve the rigor and quality of 
history education have been an important element of 
the standards-based educational reform movement of 
recent years. This emphasis on high expectations has 
been demonstrated by the rapid increase in the 
number of states that have, within the last 10 years, 
established content standards for history or social 
studies; between 1995 and 2000 the number of states 
with such standards increased from 20 to 46. 1 The 
renewed interest in history instruction underscores 
the need for accurate information about what 
students know and can do in U.S. history. 
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1 Council of Chief State School Officers. (2000). Key state education policies on K-12 
education: 2000. (table 13, p. 23). Washington, DC: Author. 
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Overview of the 2001 
National Assessment of 
Educational Progress 

For over 30 years, the National Assessment 
of Educational Progress (NAEP) has been 
authorized by Congress to collect, analyze, 
and report reliable and valid information 
about what American students know and 
can do in core subject areas. NAEP assesses 
the performance of public and nonpublic 
school students in grades 4, 8, and 12. In 
2001, student performance in U.S. history 
and geography was assessed at all three 
grades. This report deals only with the 
results of the U.S. history assessment. 

All NAEP assessments are based on 
content frameworks developed through a 
national consensus process. The NAEP 
2001 U.S. history assessment was the 
second administration of an assessment 
based on the NAEP U.S. Elistory Framework , 
which was originally developed for the 
1994 assessment. 2 In both 1994 and 2001, 
assessments based on the framework were 
administered to national samples of fourth-, 
eighth-, and twelfth-graders. 

This report describes the results of the 
2001 U.S. history assessment at grades 4, 8, 
and 12 and compares results in 2001 to 
those in 1994. Comparisons across assess- 
ment years are possible because the assess- 
ments were developed under the same 



basic framework and share a common set 
of U.S. history questions. In addition, the 
populations of students were sampled and 
assessed using comparable procedures. 

The U.S. History Framework 

Although U.S. history was assessed by 
NAEP in 1986 and 1988, a rigorous new 
NAEP U.S. History Framework was 
developed for the 1994 assessment. The 
new framework provided the operational 
specifications for both the 1994 and 2001 
assessments. The development of the 
framework was managed by the Council of 
Chief State School Officers (CCSSO) 
under the direction of the National Assess- 
ment Governing Board (NAGB). Approxi- 
mately 50 professional historians, educators, 
administrators, and other interested indi- 
viduals worked to achieve consensus on the 
general goals as well as the specific lan- 
guage of the framework. In addition , 
several hundred educational experts and 
interested members of the public contrib- 
uted to the process, either by participating 
in public hearings or by reviewing drafts. 
The framework document produced by 
this consensus process called for the assess- 
ment of a broad range of outcomes. It 
represented an ambitious vision both of 
what students should know and be able to 
do in U.S. history, and of the ways in which 
those competencies should be tested. 



2 National Assessment Governing Board. (1993). U.S. history framework for the 1994 National Assessment of Educational 
Progress. Washington, DC: Author. 
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The framework is organized around 
three concepts or dimensions: major 
themes of U.S. history, chronological 
periods, and ways of knowing and thinking 
about U.S. history with the four themes 
providing the core organizing structure of 
the framework. The themes were intended 
to ensure that all major branches of 



historical study were covered and that 
emphasis on various areas was balanced. 
The themes are also used to define the 
subscales that make up the NAJEP U.S. 
history composite scale. (See appendix A 
for more information on how the scale was 
constructed.) Figure 1.1 provides descrip- 
tions of each theme. 





Change and Continuity in American Democracy : Ideas , Institutions , Practices , and Controversies 

This theme concerns the development of American political democracy from colonial times to 
the present. It covers political events that shaped American democracy, such as the American 
Revolution, the Civil War, the fight for civil rights, as well as the core ideas and principles that 
underlie our institutions. This theme covers students’ knowledge of the founding of the nation, 
the writing of the Constitution, and other fundamental components of the nation's political 
history. At the same time, it calls for evaluating students' understanding of the role that 
major political ideas and conflicts have played at different points in our history. 



The Gathering and Interactions of Peoples, Cultures , and Ideas 

This theme is broadly defined because it covers a vast component of U.S. history: the 
interactions among the peoples and cultures of many countries, racial and ethnic groups, and 
religious traditions that have contributed to the development of American society. This theme 
covers immigration, cultural developments, patterns of social organization, and changing 
roles of men and women. 



Economic and Technological Changes and Their Relation to Society Ideas , and the Environment 

This theme focuses on the economic history of the nation and its development from a rural, 
agricultural society to an urban, industrialized superpower. It also covers the roles of 
geography and of developments in science and technology in bringing about socio-economic 
change. 



The Changing Role of America in the World 

This theme calls for coverage of the many factors — political ideas, economic interests, public 
opinion — that have shaped American foreign policy. It also addresses specific interactions 
between the United States and other nations and domestic consequences of developments in 
foreign policy. 



SOURCE: National Assessment Governing Board. U.S. History Framework for the 1994 and 2001 National Assessment of Educational Progress. 
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Eight periods provide chronological 
structure for the many issues included in 
the four themes. These periods focus 
attention on several major eras of U.S. 
history. They overlap at some points 
because they were conceived to ensure 
thorough coverage of major trends and 
events. The historical periods are not used 
as subscales, but rather were used in the 
assessment development process to ensure 
appropriate chronological coverage. The 
periods are as follows: 

□ Three Worlds and Their 
Meeting in the Americas 
(Beginnings to 1607) 

□ Colonization, Settlement, 
and Communities 
(1607 to 1763) 

□ The Revolution and the New Nation 
(1763 to 1815) 



□ Expansion and Reform 
(1801 to 1861) 

□ Crisis of the Union: 

Civil War and Reconstruction 
(1850 to 1877) 

□ The Development of Modern America 
(1865 to 1920) 

□ Modern America and the World Wars 
(1914 to 1945) 

□ Contemporary America 
(1945 to Present) 

The percentages of assessment time 
allotted to each theme and period de- 
scribed in the framework are presented in 
tables 1.1 and 1.2, respectively. It should be 
noted that these percentages vary some- 
what from the targeted distribution. (See 
appendix A, Table A. 1, for a comparison of 
the actual and targeted distributions.) 



Distribution of assessment time across historical themes, grades 4, 8, and 12: 2001 



Change and The Gathering Economic and The Changing 

Continuity in and Interactions Technological Role of 





American Democracy: 
Ideas, Institutions, 
Practices, and 
Controversies 


of Peoples, Cultures, 

and Ideas 


Changes and Their 
Relation to Society, 
Ideas, and the 

Environment 


America 

in the World 


Grade 4 


25% 


32% 


32% 


12% 


Grade 8 


30% 


32% 


25% 


13% 


Grade 12 


28% 


26% 


22% 


25% 



NOTE: Percentages may not add to 100 due to rounding. 

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 2001 U.S. History Assessment. 
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Distribution of assessment time across historical periods, grades 4, 8, and 12:2001 





Beginnings 

to 

1607 


1607 

to 

1763 


1763 

to 

1815 


1801 

to 

1861 


1850 

to 

1877 


1865 

to 

1920 


1914 

to 

1945 


1945 

to 

Present 


Ho 

period* 














Grade 4 


13% 


14% 


j 11% 


: 14% 


9% 


13% 


7% 


10% 


9% 


Grade 8 


3% 


| 

7% 

I 


18% 


9% 


8% 


18% 


14% 


10% : 

j 


12% 


Grade 12 


1% 


8% 


9% 


| ’ 

1 

11% 

i 

i 


7% 


\ I 

11% 


32%* 


i 

20% 


1% 



NOTE: Percentages may not add to 100 due to rounding. 

* A number of questions (especially at grades 4 and 8) assessed general historical skills and could not be appropriately classified in any given period. 

* This high percentage is largely a function of the 50-minute theme block, which focuses on the World War II homefront. If this block were excluded from 
calculations, questions covering this period would make up 22% of the assessment at grade 12. 

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 2001 U.S. History Assessment. 



The framework also considers ways of 
knowing and thinking about U.S. history. 
These are divided into two general cogni- 
tive domains that were used as a guide in 
exercise development. The two domains 
and their definitions are as follows: 

D Historical Knowledge and Perspective 
This domain includes knowing and 
understanding people, events, concepts, 
themes, movements, contexts, and historical 
sources; sequencing events; recognizing 
multiple perspectives and seeing an era or 
movement through the eyes of different 
groups; and developing a general 
conceptualization of U.S. history. 

□ Historical Analysis and Interpretation 
This domain includes explaining issues, 
identifying historical patterns; establishing 
cause-and-effect relationships; finding value 
statements; establishing significance; apply- 
ing historical knowledge; weighing evi- 
dence to draw sound conclusions; making 




defensible generalizations; and rendering 
insightful accounts of the past. 

U.S. History Assessment 
instruments 

As the only federally authorized ongoing 
assessment of U.S. history achievement, the 
NAEP assessment must reflect the spirit of 
the framework as well as the specifications 
provided by it. In order to achieve those 
goals, the assessment development process 
involves stages of review by teachers and 
teacher educators, state officials, and mea- 
surement experts. All components of the 
assessment are evaluated for curricular 
relevance, developmental appropriateness, 
and fairness. Final approval of NAEP test 
questions is given by the National Assess- 
ment Governing Board. A list of the U.S. 
history development committee members 
for the 2001 assessment is provided in 
appendix C. 
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The 2001 U.S. history assessment book- 
lets at grades 4 , 8, and 12 generally con- 
tained three or four sections: a set of 
general background questions, a set of 
subject-specific background questions 
dealing largely with the student s use of 
technology, and one or two sets, or 
“blocks,” of cognitive questions assessing 
knowledge and skills in U.S. history. At 
grades 8 and 12, students were given either 
two 25-minute blocks or one 50-minute 
block. At grade 4, however, only 25- 
minute blocks were used. 

At grade 4 a total of six sections, or 
“blocks”, of cognitive questions were given, 
while at grades 8 and 12 nine blocks were 
administered. In addition to the cognitive 
questions, each assessment booklet also 
included a set of background questions that 
asked students to give information about 
their school practices, such as the frequency 
with which they use computers at school 
for social studies, do research projects using 
a CD or the Internet, and write reports. 
The assessment time for each grade was 50 
minutes plus the 10—15 minutes needed to 
complete the background questions. 

Each block of cognitive questions 
consisted of both multiple-choice and 
“constructed-response” questions. (“Con- 
structed-response” is the term used to 
describe test questions in which students 
write a response, as distinct from multiple- 
choice questions, in which students choose 
an answer from one of several options.) 
Typically, a block will contain about 16—18 
questions, but there is considerable varia- 
tion depending on the balance between 
multiple-choice and constructed-response 
questions. Overall, more than 50 percent of 



student assessment time was devoted to the 
latter question type. Two types of con- 
structed-response questions were used: 

□ short constructed-response questions 
that required students to provide an- 
swers, usually in response to a text or 
visual stimulus, in one or two sentences; 
and 

□ extended constructed-response questions 
that required students to provide answers 
of a paragraph or more in length 

The 50-minute blocks administered at 
grades 8 and 12 included questions focus- 
ing on a particular theme, and included 
extended constructed-response questions 
requiring students to synthesize elements 
from various primary sources. The total 
number of test questions used in grades 4, 

8, and 12 were 94, 145, and 154, respec- 
tively. Each student answered only a small 
portion of the total number of questions. 
Additional information about the design of 
the 2001 U.S. history assessment is pre- 
sented in appendix A. 

Description of School 
and Student Samples 

The NAEP 2001 U.S. history assessment 
included representative samples of both 
public and nonpublic schools. Approxi- 
mately 7,000 fourth-graders, 11,000 
eighth-graders, and 11,000 twelfth-graders 
were assessed. The number of schools in 
the reporting sample were 365 at grade 
four, 369 at grade eight, and 374 at grade 
twelve. Each selected school that partici- 
pated in the assessment and each student 
assessed represent a portion of the popula- 
tion of interest. For additional information 
on sample sizes and participation rates, see 
appendix A. 
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This report contains two different sets of 
national results based on two reporting 
samples that differed in terms of whether 
or not accommodations were made avail- 
able to special-needs students. The national 
results presented in chapters 2, 3, 5, and 6 
of this report are based on a nationally 
representative sample that included special- 
needs students only if they could be as- 
sessed meaningfully without accommoda- 
tions. These results can be compared to 
those from 1994, because accommodations 
were also not made available in that assess- 
ment year. Chapter 4 presents a second set 
of national results from 2001 for a repre- 
sentative sample that includes the perfor- 
mance of students who required and were 
provided with accommodations (e.g., 
bilingual dictionary, extended time, small 
group testing). No comparison of these 
results to those from 1994 can be made 
because of the inclusion of these accom- 
modated special-needs students. 

In the sample that did not permit ac- 
commodations, 7 percent of fourth-graders, 
8 percent of eighth-graders, and 4 percent 
of twelfth-graders were excluded from the 
U.S. history assessment in 2001. School staff 
familiar with these students made the 
determination that these students could not 
be assessed meaningfully without accom- 
modations, because of their disability and/ 
or limited English proficiency. In 1994, 5 
percent at both the fourth and eighth 
grades, and 3 percent at the twelfth grade 
were excluded. Additional information 
regarding exclusion rates is also provided in 
appendix A. 



Reporting the 
Assessment Results 

Student performance on the NAEP U.S. 
history assessment is presented in two ways: 
as average scores on the NAEP U.S. history 
scale, and in terms of the percentage of 
students attaining NAEP U.S. history 
achievement levels. The average scale 
scores are a measure of students’ perfor- 
mance on the assessment. The achieve- 
ment level results indicate the degree to 
which student performance meets expecta- 
tions of what they should know and be 
able to do. 

Average scale score results are presented 
on the NAEP U.S. history composite scale, 
which ranges from 0-500. Students’ re- 
sponses on the NAEP 2001 U.S. history 
assessment were analyzed to determine the 
percentages of students that responded 
correctly to each multiple-choice question 
and the percentages of students that re- 
sponded at each score level for the con- 
structed-response questions. Scales that 
summarize results for each of the four 
themes described earlier were created. The 
composite scale is a weighted average of 
the separate subscales for the four themes. 
The weight for each theme corresponds to 
the theme’s relative importance in the 
NAEP U.S. history framework. A full 
description of NAEP scale procedures can 
be found in the forthcoming NAEP 2001 
Technical Report. 

Achievement-level results are presented 
in terms of U.S. history achievement levels 
as authorized by the NAEP legislation and 
adopted by the National Assessment 
Governing Board. 3 For each grade tested. 



3 No Child Left Behind Act of 2001 : Reauthorization of the Elementary and Secondary Education Act. Pub L. No 
107-110 (H.R. 1). 

National Assessment of Educational Progress Improvement Act of 1 988. Pub. L. No. 1 00-297, 20, U.S.C. 1211. 
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NAGB has adopted three achievement 
levels: Basic , Proficient , and Advanced. For 
reporting purposes, the achievement-level 
cut scores are placed on the U.S. history 
scale, resulting in four ranges: below Basic , 
Basic , Proficient , and Advanced. 

The Setting of 
Achievement Levels 

The 1988 NAEP legislation that created 
the National Assessment Governing Board 
directed the Board to identify “appropriate 
achievement goals... for each subject area” 
that NAEP measures. 4 The 2001 NAEP 
reauthorization reaffirmed many of the 
Board s statutory responsibilities, including 
developing “appropriate student achieve- 
ment levels for each grade or age in each 
subject area to be tested . . . ” D In order to 
follow this directive and achieve the man- 
date of the 1988 statute to “improve the 
form and use of NAEP results,” NAGB 
undertook the development of student 
performance standards called “achievement 
levels.” Since 1990 the Board has adopted 
achievement levels in mathematics, reading, 
U.S. history, geography, science, writing, 
and civics. 



The Board defined three levels for each 
grade: Basic , Proficient , and Advanced. The 
Basic level denotes partial mastery of the 
knowledge and skills that are fundamental 
for proficient work at a given grade. The 
Proficient level represents solid academic 
performance. Students reaching this level 
demonstrate competency over challenging 
subject matter. The Advanced level pre- 
sumes mastery of both the Basic and 
Proficient levels. Figure 1.2 presents the 
policy definitions of the achievement levels 
that apply across all grades and subject 
areas. The policy definitions guided the 
development of the U.S. history achieve- 
ment levels, as well as the achievement 
levels established in all other subject areas. 
Adopting three levels of achievement for 
each grade signals the importance of 
looking at more than one standard of 
performance. The Board believes, however, 
that all students should reach the Proficient 
level: the Basic level is not the desired goal, 
but rather represents partial mastery that is 
a step toward Proficient. 




Basic 



This level denotes partial mastery of prerequisite knowledge and skills that are 
fundamental for proficient work at each grade. 



Proficient 



This level represents solid academic performance for each grade assessed. Students 
reaching this level have demonstrated competency over challenging subject matter, 
including subject-matter knowledge, application of such knowledge to real-world 
situations, and analytical skills appropriate to the subject matter. 



Advanced 

v 



This level signifies superior performance. 



SOURCE: National Assessment Governing Board. U.S. History Framework for the 1994 and 2001 National Assessment of Educational Progress. 




4 National Assessment of Educational Progress Improvement Act of 1988. Pub. L. No. 100-297, 20, U.S.C. 1211. 

5 No Child Left Behind Act of 2001 : Reauthorization of the Elementary and Secondary Education Act. Pub. L. No. 
107-1 10 (H.R. 1). 
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The achievement levels in this report 
were adopted by the Board based on a 
standard-setting process designed and 
conducted under a contract with ACT, Inc. 
To develop these levels, ACT convened a 
cross section of educators and interested 
citizens from across the nation and asked 
them to judge what students should know 
and be able to do relative to a body of 
content reflected in the NAEP framework 
for U.S. history. This achievement-level- 
setting process was reviewed by a variety of 
individuals including policymakers, repre- 
sentatives of professional organizations, 
teachers, parents, and other members of the 
general public. Prior to adopting these 
levels of student achievement, NAGB 
engaged a large number of persons to 
comment on the recommended levels and 
to review the results. 

The results of the achievement-level- 
setting process, after NAGB’s approval, 
became a set of achievement-level descrip- 
tions and a set of achievement-level cut 
points on the 0-500 NAEP U.S. history 
scale. The cut points are the scores that 



define the boundaries between below Basic , 
Basic , Proficient , and Advanced performance 
at grades 4, 8, and 12. The Board estab- 
lished these U.S. history achievement levels 
based upon the U.S. history content frame- 
work. 

AchSevement-Level 
Descriptions for Each Grade 

Specific definitions of the Basic, Proficient, 
and Advanced U.S. history achievement 
levels for grades 4, 8, and 12 are presented 
in figures 1.3 through 1.5. As noted previ- 
ously, the achievement levels are cumula- 
tive. Therefore, students performing at the 
Proficient level also display the competencies 
associated with the Basic level, and students 
at the Advanced level also demonstrate the 
skills and knowledge associated with both 
the Basic and the Proficient levels. For each 
achievement level listed in figures 1.3 
through 1.5, the scale score that corre- 
sponds to the beginning of that level is 
shown in parentheses. For example, in 
figure 1.3 the scale score of 243 corre- 
sponds to the beginning of the grade 4 
Proficient level of achievement. 
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Basic Fourth-grade students performing at the Basic level should be able to identify and describe a 

(195) few of the most familiar people, places, events, ideas, and documents in American history. 

They should be able to explain the reasons for celebrating most national holidays, have some 
familiarity with the geography of their own state and the United States, and be able to express 
in writing a few ideas about a familiar theme in American history. 



Proficient 

(243) 



Fourth-grade students performing at the Proficient level should be able to identify, describe 
and comment on the significance of many historical people, places, ideas, events, and 
documents. They should interpret information from a variety of sources, including texts, maps, 
pictures, and timelines. They should be able to construct a simple timeline from data. These 
students should recognize the role of invention and technological change in history. They 
should also recognize the ways in which geographic and environmental factors have influ- 
enced life and work. 



Advanced 

(276) 



Fourth-grade students performing at the Advanced level should have a beginning 
understanding of the relationship between people, places, ideas, events, and documents. 
They should know where to look for information, including reference books, maps, local 
museums, interviews with family and neighbors, and other sources. They should be able to 
use historical themes to organize and interpret historical topics, and to incorporate insights 
from beyond the classroom into their understanding of history. These students should 
understand and explain the role of invention and technological change in history. They should 
also understand and explain the ways in which geographic and environmental factors have 
influenced life and work. 



SOURCE: National Assessment Governing Board. U.S. History Framework for the 1994 and 2001 National Assessment of Educational Progress. 
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Basie 


Eighth-grade students performing at the Basic level should be able to identify and place in 


(252) 


context a range of historical people, places, events, ideas, and documents. They should be 
able to distinguish between primary and secondary sources. They should have a beginning 
understanding of the diversity of the American people and the ways in which people from a 
wide variety of national and cultural heritages have become part of a single nation. Eighth- 
grade students at the Basic level should also have a beginning understanding of the 
fundamental political ideas and institutions of American life and their historical origins. They 
should be able to explain the significance of some major historical events. 


Proficient 


Eighth-grade students performing at the Proficient level should be able to explain the 


(294) 


significance of people, places, events, ideas, and documents, and to recognize the connection 
between people and events within historical contexts. They should understand and be able to 
explain the opportunities, perspectives and challenges associated with a diverse cultural 
population. They should incorporate geographic, technological, and other considerations in 
their understanding of events and should have knowledge of significant political ideas and 
institutions. They should be able to communicate ideas about historical themes while citing 
evidence from pri mary and secondary sources to support their conclusions. 


Advanced 


Eighth-grade students performing at the Advanced level should recognize significant themes 


( 327) 


and movements in history and begin to understand particular events in light of these themes 
and movements. They should have an awareness of continuity and change over time and be 
able to draw relevant analogies between past events and present-day situations. They should 
be able to frame questions about historical topics and use multiple sources to develop 
historical generalizations and interpretations. They should be able to explain the importance 
of historical themes, including some awareness of their political, social, and economic 


V 


dimensions. 



SOURCE: National Assessment Governing Board. U.S. History Framework for the 1994 and 2001 National Assessment of Educational Progress. 
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Basic Twelfth-grade students performing at the Basic level should be able to identify the 
(294) significance of many people, places, events, dates, ideas, and documents in U.S. history. They 
should also recognize the importance of unity and diversity in the social and cultural history 
of the United States, and an awareness of American’s changing relationships with the rest of 
the world. They should have a sense of continuity and change in history and be able to relate 
relevant experience from the past to their understanding of contemporary issues. They should 
recognize that history is subject to interpretation and should understand the role of evidence 
in making an historical argument. 




Proficient 

(325) 



Twelfth-grade students performing at the Proficient level should understand particular 
people, places, events, ideas, and documents in historical context, with some awareness of 
the political, economic, geographic, social, religious, technological, and ideological factors 
that shape historical settings. They should be able to communicate reasoned interpretations 
of past events, using historical evidence effectively to support their positions. Their written 
arguments should reflect some in-depth grasp of issues and refer to both primary and 
secondary sources. 



Advanced 



Twelfth-grade students achieving at the Advanced level should demonstrate a comprehensive 
understanding of events and sources of U.S. history. Recognizing that history is subject to 
interpretation, they should be able to evaluate historical claims critically in light of the 
evidence. They should understand that important issues and themes have been addressed 
differently at different times and that America's political, social, and cultural traditions have 
changed overtime. They should be able to write well-reasoned arguments on complex 
historical topics and draw upon a wide range of sources to inform their conclusions. 



SOURCE: National Assessment Governing Board. U.S. History Framework tor the 1994 and 2001 National Assessment ot Educational Progress. 
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The Trial Status of 
Achievement LeveSs 

The 2001 NAEP reauthorization law 
requires that the achievement levels be 
used on a trial basis until the Commis- 
sioner of Education Statistics determines 
that the achievement levels are “reasonable, 
valid, and informative to the public.” 6 Until 
that determination is made, the law re- 
quires the Commissioner and the Board to 
state clearly the trial status of the achieve- 
ment levels in all NAEP reports. 

In 1993, the first of several congression- 
ally mandated evaluations of the achieve- 
ment level setting process concluded that 
the procedures used to set the achievement 
levels were flawed and that the percentage 
of students at or above any particular 
achievement level cutpoint may be under- 
estimated. 7 Others have critiqued these 
evaluations, asserting that the weight of the 
empirical evidence does not support such 
conclusions. 8 

In response to the evaluations and 
critiques, NAGB conducted an additional 
study of the 1992 reading achievement 



levels before deciding to use those reading 
achievement levels for reporting 1994 
NAEP results. 9 When reviewing the find- 
ings of this study, the National Academy of 
Education (NAE) Panel expressed concern 
about what it saw as a “confirmatory bias” 
in the study and about the inability of this 
study to “address the panels perception that 
the levels had been set too high.” 10 In 1997, 
the NAE Panel summarized its concerns 
with interpreting NAEP results based on 
the achievement levels as follows: 

First , the potential instability of the levels 
may intefere with the accurate portrayal of 
trends. Second , the perception that few American 
students are attaining the higher standards we 
have set for them may defect attention to the 
wrong aspects of education reform. The public has 
indicated its interest in benchmarking against 
international standards, yet it is noteworthy that 
when American students pe formed very well on 
a 1991 international reading assessment, these 
results were discounted because they were 
contradicted by poor peformance against the 
possibly fawed NAEP reading achievement 
levels in the following yearf 



6 No Child Left Behind Act of 2001: Reauthorization of the Elementary and Secondary Education Act. Pub. L. No. 
107-110 (H.R. 1). 

7 United States General Accounting Office. (1993). Education achievement statidards: NAGB's approach yields misleading 
interpretations. U.S. General Accounting Office Report to Congressional Requestors. Washington, DC: Author. 

National Academy of Education. (1993). Setting performance standards for achievement: A report of the National Academy 
of Education Panel on the evaluations of the NAEPTrial State Assessment: An evaluation of the 1992 achievement levels. 
Stanford, CA: Author. 

8 Cizek, G. (1993). Reactions to National Academy of Education report. Washington, DC: National Assessment Governing 
Board. 
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American College Testing. (1995). NAEP reading revisited: An evaluation of the 1992 achievement level descriptions. 
Washington, DC: National Assessment Governing Board. 

National Academy of Education. (1996). Reading achievement levels. In Quality and utility :The 1994 Trial State 
Assessment in reading. The fourth report of the National Academy of Education Panel on the evaluation of the NAEPTrial 
State Assessment. Stanford, CA: Author. 



National Academy of Education. (1997). Assessment in transition : Monitoring the nation's educational progress (p. 99). 
Mountain View, CA: Author. 
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The National Center for Education 
Statistics and the National Assessment 
Governing Board have sought and con- 
tinue to seek new and better ways to set 
performance standards on NAEP. 12 For 
example, NCES and NAGB jointly spon- 
sored a national conference on standard 
setting in large-scale assessments, which 
explored many issues related to standard 
setting. 13 Although new directions were 
presented and discussed, a proven alterna- 
tive to the current process has not yet been 
identified. The Deputy Commissioner of 
Education Statistics and the Board con- 
tinue to call on the research community to 
assist in finding ways to improve standard 
setting for reporting NAEP results. 

The most recent congressionally man- 
dated evaluation conducted by the Na- 
tional Academy of Sciences (NAS) relied 
on prior studies of achievement levels, 
rather than carrying out new evaluations, 
on the grounds that the process has not 
changed substantially since the initial 
problems were identified. Instead, the NAS 
Panel studied the development of the 1996 
science achievement levels. The NAS Panel 
basically concurred with earlier congres- 
sionally mandated studies. The Panel 
concluded that “NAEP s current achieve- 



ment level setting procedures remain 
fundamentally flawed. The judgment tasks 
are difficult and confusing; raters’ judg- 
ments of different item types are internally 
inconsistent; appropriate validity evidence 
for the cut scores is lacking; and the process 
has produced unreasonable results.” 14 

The NAS Panel accepted the continuing 
use of achievement levels in reporting 
NAEP results on a developmental basis, 
until such time as better procedures can be 
developed. Specifically, the NAS Panel 
concluded that “....tracking changes in the 
percentages of students performing at or 
above those cut scores (or, in fact, any 
selected cut scores) can be of use in de- 
scribing changes in student performance 
over time.” 15 

The National Assessment Governing 
Board urges all who are concerned about 
student performance levels to recognize 
that the use of these achievement levels is a 
developing process and is subject to various 
interpretations. The Board and the Deputy 
Commissioner believe that the achieve- 
ment levels are useful for reporting trends 
in the educational achievement of students 
in the United States. 16 In fact, achievement 
level results have been used in reports by 
the President of the United States, the 



12 Reckase, Mark, D. (2000). 77 ic evolution of the NAEP achievement levels setting process: A summary of the research and 
development efforts conducted by ACT. Iowa City, IA: ACT, Inc. 

13 National Assessment Governing Board and National Center for Education Statistics. (1995). Proceedings of the joint 
conference on standard setting for large-scale assessments of the National Assessment Governing Board (NAGB) and the 
National Center for Education Statistics (NCES). Washington, DC: Government Printing Office. 

14 Pellegrino, J.W., Jones, L.R., St Mitchell, K.J. (Eds.). (1998). Grading the nation’s report card: evaluating NAEP and 
transforming the assessment of educational progress. Committee on the Evaluation of National Assessments of Educa- 
tional Progress, National Research Council, (p.l 82). Washington, DC: National Academy Press. 

15 Ibid., page 176. 

R Forsyth, Robert A. (2000). A description of the standard-setting procedures used by three standardized test 
publishers. In Student performance standards on the National Assessment of Educational Progress : Affirmations and 
itnprovements. Washington, DC. National Assessment Governing Board. 

Nellhaus, Jeffrey M. (2000). States with NAEP-like performance standards. In Student performance standards on the 
National Assessment of Educational Progress: Affirmations and improvements. Washington, DC: National Assessment 
Governing Board. ^ < 
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Secretary of Education, state governors, 
legislators, and members of Congress. 
Government leaders in the nation and in 
more than 40 states use these results in 
their annual reports. 

However, based on the congressionally 
mandated evaluations so far, the Deputy 
Commissioner agrees with the National 
Academy’s recommendation that caution 
needs to be exercised in the use of the 
current achievement levels. Therefore, the 
Deputy Commissioner concludes that 
these achievement levels should continue 
to be used on a trial basis and should 
continue to be interpreted with caution. 

Interpreting NAEP Results 

The average scores and percentages pre- 
sented in this report are estimates because 
they are based on samples of students rather 
than on entire populations. Moreover, the 
collection of questions used at each grade 
level is but a sample of the many questions 
that could have been asked to assess student 
knowledge of the framework content. As 
such, the results are subject to a measure of 
uncertainty, reflected in the standard error 
of the estimates. The standard errors for 
the estimated scale scores and percentages 
in this report are provided in appendix B. 



The differences between scale scores and 
between percentages discussed in the 
following chapters take into account the 
standard errors associated with the esti- 
mates. Comparisons are based on statistical 
tests that consider both the magnitude of 
the difference between the group average 
scores or percentages and the standard 
errors of those statistics. Throughout this 
report, differences between scores or 
between percentages are pointed out only 
when they are significant from a statistical 
perspective. All differences reported are 
significant at the 0.05 level with appropri- 
ate adjustments for multiple comparisons. 
The term significant is not intended to 
imply a judgment about the absolute 
magnitude or the educational relevance of 
the differences. It is intended to identify 
statistically dependable population differ- 
ences to help inform dialogue among 
policymakers, educators, and the public. 

Readers are cautioned against interpret- 
ing NAEP results in a causal sense. Infer- 
ences related to student subgroup perfor- 
mance or to the effectiveness of public and 
nonpublic schools, for example, should take 
into consideration the many socioeco- 
nomic and educational factors that may 
also impact on performance in U.S. history. 
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Overview of the 
Remaining Report 

The results in chapters 2 and 3 of this 
report are based on the set of data with no 
accommodations offered to students. 
Findings are presented for the nation and 
for all the major reporting subgroups 
included in all NAEP report cards. Com- 
parisons with results from the 1994 assess- 
ment are noted where the data permit. 

NAEP has sought to assess samples that 
are as inclusive as possible. Nevertheless, 
there has always been some exclusion of 
students with disabilities (SD) and limited 
English proficient (LEP) students who 
could not be assessed meaningfully without 
accommodations. Local school officials 
have made decisions about exclusion in 
accordance with explicit criteria provided 
by the NAEP program. In order to expand 
the proportion of students who can be 
assessed meaningfully, the NAEP program 
began in recent assessments to explore the 
use of accommodations with special-needs 
students. Chapter 4 presents an overview of 
a second set of results — those that include 
students who were provided accommoda- 
tions during the test administration. By 
including these results in the nations U.S. 
history report card, the NAEP program 
continues a phased transition toward a 
more inclusive reporting sample. Future 



assessment results will be based solely on a 
student and school sample in which ac- 
commodations are permitted. 

Chapter 5 provides sample assessment 
questions and student responses from the 
2001 assessment. Also presented in chapter 
5 are item maps that position selected 
question descriptions along the NAEP U.S. 
history scale where they are likely to be 
answered successfully by students. The 
descriptions used on these item maps focus 
on the U.S. history skill or knowledge 
needed to answer the question. Chapter 6 
examines contexts for learning U.S. history 
in terms of classroom practices and student 
variables. The data presented in both 
chapters 5 and 6 are based on the set of 
results that did not include accommodated 
special-needs students. 

This report also contains appendices that 
support or augment the results presented. 
Appendix A contains an overview of the 
NAEP U.S. history framework and specifi- 
cations, information on the national 
sample, and a more detailed description of 
the major reporting subgroups featured in 
chapters 2 and 3. Appendix B contains the 
full data with standard errors for all tables 
and figures in this report. Appendix C 
contains a list of the NAEP U.S. history 
committee members. 
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Are the nation’s 
fourth-, eighth-, 
and twelfth- 
graders making 
progress in 
U.S. history? 



Overview 

This chapter presents results for the nation from the NAEP 
2001 U.S. history assessment at grades 4, 8, and 12. Student 
performance is described in two ways: one, by average scores 
on the NAEP U.S. history scale, which ranges 
from 0 to 500; and two, in terms of the percentages of 
students who attained each of the three U.S. history 

achievement levels: Basic, Proficient, and Advanced. 
Results of the NAEP 2001 U.S. history assessment 
are compared with results from the previous 
assessment, which took place in 1994. This 
comparison is possible because the assessments share 
a common set of tasks based on the current U.S. 
history framework and because the population of 
students in both years was sampled and assessed using 
comparable procedures. The results presented in this 
chapter are based on a representative sample of 
students assessed under conditions that did not 
permit accommodations for special-needs students. 
These were the same conditions under which the 
1994 history assessment was administered, thus 
making it possible to report trends in student performance 
across the assessment years. A second set of results, reflecting 
part of a phased transition toward a more inclusive reporting 
sample in which accommodations were permitted for 
special-needs students, is presented in chapter 4. 




Overview 

Average Scale 
Score Results 

Scale Scores by 
Percentile 

Actiievement- 
Level Results 
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Average Scale Score Results 

The results of the NAEP 2001 U.S. history 
assessment show improvement in student 
performance over the 1994 results at grades 
4 and 8, but do not show a statistically 



significant change at grade 12. Figure 2.1 
presents the average U.S. history scale 
scores for the nation for fourth-, eighth-, 
and twelfth-graders attending both public 
and nonpublic schools in 1994 and 2001. 





• I 

★ Significantly different from 1994. 

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 1994 and 2001 U.S. History Assessments. 
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Scale Scores by Percentile 

One way to view students’ scale score 
performance on the U.S. history assessment 
is by looking at the scale scores attained by 
students across the performance 
distribution. The percentile indicates the 
percentage of students whose scores fell 
below a particular average score. The 



advantage of looking at the data this way is 
that it reveals changes in performance for 
both lower- and higher-performing 
students. Figure 2.2 presents the U.S. 
history scale scores for grades 4, 8, and 12 
at the 10th, 25th, 50th, 75th, and 90th 
percentiles for both the 1994 and the 2001 
assessments. 





★ Significantly different from 1994. 

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 1994 and 2001 U.S. History Assessments. 
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At grade 4, the scale scores at the lower 
percentiles (10th and 25th) were higher in 
2001 than in 1994. At the other percentiles 
for grade 4, apparent changes since 1994 
were not statistically significant. Increases in 
average scores at grade 8 were evident 
among students at both the lower percen- 
tile (25th) and the upper percentiles (75th 
and 90th). At the 10th and 50th percentiles, 
however, apparent changes since 1994 were 
not statistically significant. There were no 
statistically significant changes in average 
scores when viewed across the score distri- 
bution at grade 12. 

Achievement-Level Results 

The achievement levels are performance 
standards adopted by the National Assess- 
ment Governing Board, based on the 
collective judgements of experts about 



what students should be expected to know 
and be able to do in terms of the NAEP 
U.S. history framework. A discussion of 
the trial status of achievement levels is in 
chapter 1. 

Achievement-level results for each grade 
are presented in figure 2.3. Results are 
presented in two ways: 1) the percentage of 
students within each achievement-level 
range, and 2) the percentage of students at 
or above the Basic and at or above the 
Proficient levels. In reading figure 2.3, it is 
necessary to keep in mind that the percent- 
ages at or above specific achievement levels 
are cumulative. For example, included 
among the percentage of students at or 
above the Basic level are also those who 
have achieved the Proficient and Advanced 
levels of performance. 
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Grade 4 



Grade 8 




At or above 
Proficient 



At or above 
Basic 



1994 2001 



Advanced 

Proficient 



Basic 



Below 

Basic 



1 % 2 %* 



13% 


14% I 


-l 


61% 


m 





15%* 


17%*| 


« 


64%* 


w 





At or above 
Proficient 



At or above 
Basic 



1994 2001 



Grade 12 




At or above 
Proficient 

At or above 
Basic 



1994 2001 



★ Significantly different from 1994. 

NOTE: Percentages within each U.S. history achievement level may not add to 100, or to the exact percentages at or above achievement levels, due to rounding. 
SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 1994 and 2001 U.S. History Assessments. 
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In the 2001 U.S. history assessment, 18 
percent of fourth-graders, 17 percent of 
eighth-graders, and 11 percent of twelfth- 
graders performed at or above the Proficient 
level — identified by NAGB as the level at 
which all students should perform. Stu- 
dents’ attainment of the achievement levels 
across years generally reflects the changes in 
scale score results described in the previous 
section: improvement at some levels at 
grade 4, general improvement at grade 8, 
and no statistically significant change at 
grade 12. 



At grade 4, the percentage of students 
performing at or above Basic in 2001 was 
higher than in 1994. At grade 8, there was 
improvement across the levels compared to 
the previous assessment, with the percent- 
ages of students performing at or above 
Basic , at or above Proficient , and at Advanced 
increasing from 1994 to 2001. At grade 12, 
however, there was no statistically signifi- 
cant change in the percentages of students 
performing at each level. As in 1994, only 
a small percentage of students at each grade 
performed at the Advanced level, with 2 
percent at grades 4 and 8, and 1 percent at 
grade 12 attaining that level. Fifty-seven 
percent of twelfth-graders were below the 
Basic achievement level in 2001. 
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3 Average Scale Scores and 
Achievement-Level Results 
for Selected Subgroups 

In addition to reporting on the performance of all students, 
NAEP also provides results for various subgroups of students 
at each grade. Examining subgroup results provides insight, 
not only into how these groups of students performed in 
comparison to one another, but also into how each group 
has progressed over time. The information presented in this 
chapter serves as a valuable indicator of the progress of 
subgroups of the students across the nation. 

Results for the NAEP 2001 U.S. history 
assessment are presented by gender, race/ethnicity, 
region of the country, parents’ highest level of 
education, type of school, type of location, and 
eligibility for the Free/Reduced-Price School Lunch 
program. For all subgroups except two — type of 
location and free/reduced-price school lunch 
eligibility — results are available from 1994, and are 
presented here for comparison with results from 2001. 

All differences reported in this chapter between 
demographic subgroups for the 2001 assessment and 
between the 2001 and the 1994 results are based on 
statistical tests that consider both the magnitude of the 
difference between the group average scores or percentages 
and the standard errors of those statistics. Differences 
between groups and between assessment years are discussed 
only if they have been determined to be statistically 
significant. Furthermore, the reader should bear in mind that 
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history? if 
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differences in performance among subgroups 
of students most likely reflect a range of 
socioeconomic and educational factors not 
addressed in this report or by NAEP. 

Gender 

Figure 3.1 presents the 1994 and 2001 
average U.S. history scale scores by gender. 
Patterns in performance among male and 
female students varied somewhat by grade. 
At grade 4 both male and female students 
had higher average scores in 2001 than in 



1994, while at grade 8 only males showed a 
gain in 2001 over 1994. The apparent gain 
for female eighth-graders was not statisti- 
cally significant. At grade 12, there was no 
statistically significant change in the perfor- 
mance of male and female students from 
one assessment to the next. At all three 
grades in 2001, there was no statistically 
significant difference between the perfor- 
mance of males and females. 



Figure 3.1 


H0[)90HDa® 


National Scale Score 
Results by Gender 


j- * ■ 



Grade 4 Grade 8 Grade 12 




★ Significantly different from 1994. 

NOTE: Italicized scale score values indicate that two or more groups had the same rounded average score. The average scale scores, when rounded, were the 
same for male and female students at grade 4 in 2001 and grade 8 in 1994. 

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 1994 and 2001 U.S. History Assessments. 
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The performance of subgroups on the 
U.S. history assessment can also be com- 
pared by determining whether a difference 
or “gap” exists between groups’ average 
scores and, if it does, whether that gap 
increases or decreases between assessment 



years. As stated previously, no significant 
difference between male or female average 
scores was observed in 2001. Figure 3.2 
also shows that there was no statistically 
significant change between 1994 and 2001 
in these small and nonsignificant gender gaps 




Male-Female 

Grade 4 Grade 8 Grade 12 



2001 -1® 


2001 


•2 2001 


• 2 


1994 -2»- 


1994 #« 


* 1994 


■•3 


-10 ( 


) 10 20 30 40 -10 ( 

Score Differences 


) 10 20 30 40 -10 ( 

Score Differences 


) 10 20 30 40 

Score Differences 



# Difference is between *0.5 and 0.5. 

NOTE: Score differences are calculated based on differences between unrounded average scale scores. 



SOURCE: U.S. Department of Education, Office of Educational Research and 
Educational Progress (NAEP), 1994 and 2001 U.S. History Assessments. 

The percentages of male and female 
students at or above the U.S. history 
achievement levels and within each 
achievement level range are presented in 
figure 3.3. Achievement-level results for 
males and females at both grades 4 and 12 
showed no statistically significant increases 
or decreases since 1994. At grade 8, how- 
ever, the percentages of male students at or 
above Basic , at or above Proficient , and at 
Advanced were all higher in 2001 than in 



0 




Improvement, National Center for Education Statistics, National Assessment of 

1994, while any apparent changes in the 
percentages of females at or above any of 
the achievement levels were not statistically 
significant. A comparison of the percent- 
ages of male and female students at or 
above the Basic and Proficient levels in 2001 
shows no difference at grade 4, but does 
show a higher percentage of males than 
females at or above Proficient at grade 8, and 
a higher percentage of males than females 
at or above Basic at grade 12. 
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★ Significantly different from 1994. 

NOTE: Percentages within each U.S. history achievement-level range may not add to 100, or to the exact percentages at or above achievement levels, due to rounding. 
SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 1994 and 2001 U.S. History Assessments. 
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Race/Ethnicity 

Students participating in the U.S. history 
assessment were asked to indicate which of 
the following racial/ ethnic subgroups best 
described them — White, Black, Hispanic, 
Asian/Pacific Islander, or American Indian 
(including Alaska Native). Figure 3.4 
presents average scale scores for students by 
these subgroups at grades 4, 8, and 12. 

At grade 4, both White students and 
Black students had higher average scores in 
2001 than in 1994. At grade 8, White 
students showed a gain since 1994, and at 
grade 12 Hispanic students had higher 
average scores in 2001, compared to 1994. 
No other changes were statistically 
significant. 

In 2001, differences in performance by 
racial/ethnic subgroup continue to be 



evident at all three grades. On average, 
White students had higher scores than 
Black, Hispanic, and American Indian 
students at all three grades. Asian/Pacific 
Islander students had higher average scores 
than Black and Hispanic students at all 
three grades. White fourth-grade students 
had higher average scores than Asian/ 
Pacific Islander fourth-graders. These 
differences should be interpreted with 
caution. The average score of a selected 
subgroup does not represent the entire 
range of performance within that group. 
Furthermore, differences between groups 
of students cannot be attributed solely to 
group identification. A complex array of 
educational and social factors interacts to 
affect average student performance. 



1 National Scale Score 
| Results by Race/Ethnicity 



Grade 4 




A American Indian 
★ Significantly different from 1994. 



Grade 8 




A American Indian 



Grade 12 




A American Indian 



NOTE: Italicized scale score values indicate that two or more groups had the same rounded average score. The average scale scores, when rounded, were the 
q ne for Hispanic and Black students at grade 8 in 2001. 

I D I f JRCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of 
icational Progress (NAEP), 1994 and 2001 U.S. History Assessments. 
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Scale score differences between White 
students and Black students and between 
White students and Hispanic students are 
presented in figure 3.5. Results from the 
2001 U.S. history assessment reflect a 



narrowing of the score gap between White 
students and Black students at grade 4, and 
between White students and Hispanic 
students at grade 12. 



Figure 3.5 

!H 

j National Scale Score 
P Differences by Race/ 
Ethnicity ; 



Grade 4 



White-Black 

Grade 8 



Grade 12 



O 

ERIC 






Score Differences 



Score Differences 



Score Differences 



Grade 4 




White— Hispanic 

Grade 8 




Grade 12 




Score Differences 



Score Differences 



Score Differences 



★ Significantly different from 1994. 

NOTE: Score differences are calculated based on differences between unrounded average scale scores. 

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 1994 and 2001 U.S. History Assessments. 



Achievement-level results for the racial/ 
ethnic subgroups are presented in figure 
3.6a, b, and c. While there have been some 
overall achievement gains since 1994 at 
grades 4 and 8, not all groups show im- 
provement. At grade 4, both White stu- 
dents and Black students had higher per- 
centages at or above Basic in 2001 com- 
pared to 1994. At grade 8, however, only 
White students showed an increase in the 
percentages at or above Proficient and at 



Advanced. At grade 12, none of the appar- 
ent changes between 1994 and 2001 in the 
percentages of students at or above any of 
the history achievement levels were statisti- 
cally significant. 

Comparing the subgroups' performance 
in 2001 shows higher percentages of 
White students and Asian/Pacific Islander 
students at or above the Basic and Prof dent 
levels than Black and Hispanic students at 
all three grades. 
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NOTE: Percentages within each U.S. history achievement-level range may not add to 100, or to the exact percentages at or above achievement levels, due to rounding. 
SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 1994 and 2001 U.S. History Assessments. 
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★ Significantly different from 1994. 

# Percentage is between 0.0 and 0.5. 

NOTE: Percentages within each U.S. history achievement-level range may not add to 100, or to the exact percentages at or above achievement levels, due to rounding. 
SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 1994 and 2001 U.S. History Assessments. 
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NOTE: Percentages within each U.S. history achievement-level range may not add to 100, or to the exact percentages at or above achievement levels, due to rounding. 
SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 1994 and 2001 U.S. History Assessments. 
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Region of the Country 

NAEP assessments traditionally provide 
results for four regions of the country: 
Northeast, Southeast, Central, and West. 
Appendix A (see page 129) contains a 
description of the states and other jurisdic- 
tions that make up each region. 

Scale score results by region are pre- 
sented in figure 3.7. Although overall gains 
in student performance were observed at 
grades 4 and 8, not all regions showed 
increases. At grade 4, only the Northeast 
region showed a gain in the U.S. history 
average score since 1994, while at grade 8, 
the only increase occurred in the Southeast 
region. None of the other apparent 
changes between 1994 and 2001 in re- 



gional average scores were statistically 
significant. Regional results at grade 12 
were consistent with the overall national 
results that did not show a change in 
students’ performance from 1994 to 2001. 

Some differences in performance be- 
tween regions of the country were evident 
in 2001. At both grades 4 and 8, students 
in the Northeast, Southeast, and Central 
regions all had higher average scores than 
students in the West, and students in the 
Central region outperformed their peers in 
the Southeast on average. At grade 12, 
none of the apparent differences in average 
scores among the four regions were statisti- 
cally significant. 




Grade 4 Grades Grade 12 




★ Significantly different from 1994. 

NOTE: Italicized scale score values indicate that two or more groups had the same rounded average score. The average scale scores, when rounded, were the 
same for Northeast and Central regions at grade 8 in 1994 and at grade 12 in 2001. 

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 1994 and 2001 U.S. History Assessments. 
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Achievement-level results for the four 
regions are displayed in figure 3.8a, b, and c 
by grade. The only gains occurred in the 
Southeast region at grade 8, where the 
percentage both at or above Basic and at or 
above Proficient increased in 2001 compared 
to 1994. 

As with the scale score results by region, 
some differences between regions in the 
percentages of students at or above the 
different achievement levels were evident 
in 2001. A higher percentage of fourth- 
grade students in the Northeast and Cen- 



tral regions were at or above both the Basic 
and Proficient levels than in the West, and a 
higher percentage of fourth-graders in the 
Central region were at or above Basic than 
in the Southeast. There were also higher 
percentages of eighth-grade students in the 
Northeast and Central regions at or above 
Basic and Proficient than in the West. In 
addition, the percentage of eighth-graders 
at or above Basic was higher in the North- 
east and Central region than in the South- 
east, and higher in the Southeast than in 
the West. 
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NOTE: Percentages within each U.S. history achievement-level range may not add to 100, or to the exact percentages at or above achievement levels, due to rounding. 
SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of 
cational Progress (NAEP), 1994 and 2001 U.S. History Assessments. 
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★ Significantly different from 1994. 

# Percentage is between 0.0 and 0.5. 

NOTE: Percentages within each U.S. history achievement-level range may not add to 100, or to the exact percentages at or above achievement levels, due to rounding. 
SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 1994 and 2001 U.S. History Assessments. 
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# Percentage is between 0.0 and 0.5. 

NOTE: Percentages within each U.S. history achievement-level range may not add to 100, or to the exact percentages at or above achievement levels, due to rounding. 
SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 1994 and 2001 U.S. History Assessments. 
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Parents’ Highest Level 
of Education 

Eighth- and twelfth-grade students who 
participated in the NAEP U.S. history 
assessment were asked to indicate the 
highest level of education completed by 
each parent. Four levels of education were 
identified: did not finish high school, 
graduated from high school, some educa- 
tion after high school, and graduated from 
college. Students could also choose the 
response, “I don’t know.” For this analysis, 
the highest education level reported for 
either parent was used. Data were not 
collected at grade 4 because in previous 
NAEP assessments fourth-graders’ re- 
sponses about their parents’ education were 
highly variable and contained a large 
percentage of “I don’t know” responses. 

The scale score results for all levels of 
student-reported parent education level are 
presented in figure 3.9. In 2001, almost 
one-half of the eighth- and twelfth-grade 



students reported at least one parent had 
graduated from college (47 and 46 percent, 
respectively), whereas only a small percent- 
age reported that their parents had not 
graduated high school (7 percent at both 
grades). Additional information on the 
percentage of students reporting parents’ 
highest level of education is available in 
appendix B. 

At grade 8, students who reported that at 
least one parent graduated from college 
had higher average scores in 2001 than in 
1994. At grade 12, there was an increase in 
the average scores of students who reported 
that neither parent finished high school. 

The 2001 results indicate that, overall, there 
was a clear positive relationship between 
parent education level and the performance 
of both eighth- and twelfth-graders on the 
U.S. history assessment. At both grades, 
the higher the parental education level 
reported, the higher the average score 
attained. 
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□ Some education after high school □ Some education after high school 

3 Graduated high school H Graduated high school 

® Less than high school 0 Less than high school 

A Unknown A Unknown 

★ Significantly different from 1994. 

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 1994 and 2001 U.S. History Assessments. 



Achievement-level results across years by 
level of parental education are presented in 
figure 3.10a and b. 

There were higher percentages of 
eighth-graders at or above Proficient and at 
Advanced in 2001 among students who 
reported that at least one parent graduated 
from college. None of the other changes in 
achievement level results by level of paren- 
tal education were statistically significant. 



As with the average scale score results, 
the 2001 achievement-level results show 
higher percentages of eighth- and twelfth- 
grade students at or above both the Basic 
and Proficient levels among students whose 
parents graduated from college than among 
those who reported parents having lower 
levels of education. 
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★ Significantly different from 1994. 
ft Percentage is between 0.0 and 0.5. 

NOTE: Percentages within each U.S. history achievement-level range may not add to 100, or to the exact percentages at or above achievement levels, due to rounding. 
SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 1994 and 2001 U.S. History Assessments. 
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# Percentage is between 0.0 and 0.5. 

NOTE: Percentages within each U.S. history achievement-level range may not add to 100, or to the exact percentages at or above achievement levels, due to rounding. 
SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 1994 and 2001 U.S. History Assessments. 
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Type of School 

The schools that participate in the NAEP 
assessment are classified as either public or 
nonpublic. A further distinction is then 
made within the nonpublic classification 
between nonpublic schools that are Catho- 
lic and other nonpublic schools. In 2001, as 
in previous NAEP assessments, fourth-, 
eighth, and twelfth-grade students attend- 
ing various types of nonpublic schools had 
higher average scores than did their peers 
attending public schools. Readers are, 
however, cautioned against making assump- 
tions about the comparative quality of 
instruction in public and nonpublic 
schools. Socioeconomic and sociological 
factors that may affect student performance 
should be considered when interpreting 
these results. 



Average U.S. history scale scores by type 
of school are presented in figure 3.11 and 
show higher average scores in 2001 than in 
1994 among fourth- and eighth-grade 
students attending public schools. Fourth- 
graders attending Catholic nonpublic 
schools also had higher average scores in 
2001 than in 1994. None of the apparent 
changes seen at grade 12 were statistically 
significant. 

Comparisons of scale score results 
between the types of schools in 2001 show 
students at Catholic nonpublic schools 
outperforming public school students at all 
three grades, while the performance of 
students in other nonpublic schools was 
higher than that of public school students 
at grades 4 and 8 only. 




Grade 





© Public 
□ Nonpublic 
O Nonpublic: Catholic 
M Nonpublic: Other 



See footnotes at end of table. > 
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O Nonpublic: Catholic 
H Nonpublic: Other 



★ Significantly different from 1994. 

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 1994 and 2001 U.S. History Assessments. 
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Achievement-level results by school type 
are presented in figure 3.12a, b, and c by 
grade. At grade 4, there was a higher 
percentage of Catholic nonpublic school 
students at or above Proficient in 2001 than 
in 1994. At grade 8, there were higher 
percentages of public school students at or 
above Proficient in 2001 than in 1994. 



Comparisons of 2001 achievement-level 
results between types of schools show 
higher percentages of nonpublic school 
students at or above the Basic and Proficient 
levels than public school students at all 
three grades. There was also a. higher 
percentage of eighth-grade students in 
nonpublic schools at the Advanced level 
than in public schools. 
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★ Significantly different from 1994. 

NOTE: Percentages within each U.S. history achievement-level range may not add to 100, or to the exact percentages at or above achievement levels, due to rounding. 
SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 1994 and 2001 U.S. History Assessments. 
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★ Significantly different from 1994. 

NOTE: Percentages within each U.S. history achievement-level range may not add to 100, or to the exact percentages at or above achievement levels, due to rounding. 
SOURCE: U.S. Department of Education, Office of Educational Research and improvement, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 1994 and 2001 U.S. History Assessments. 
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NOTE: Percentages within each U.S. history achievement-level range may not add to 100, or to the exact percentages at or above achievement levels, due to rounding. 
SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 1994 and 2001 U.S. History Assessments. 
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Type of Location 

The schools from which NAEP draws its 
samples of students are classified according 
to their type of location. Based on Census 
Bureau definitions of metropolitan statisti- 
cal areas, including population size and 
density, the three mutually exclusive cat- 
egories are: central city, rural/small town, 
and urban fringe/large town. Because of 
new methods used by NCES to identify 
the type of location assigned to each school 
in the Common Core of Data, schools 
were not classified in exactly the same way 
in 2001 as in 1994. Therefore, comparisons 
between the two assessment years are not 



possible, and only the data for the 2001 
assessment are reported. More information 
on the definitions of the 2001 assessment 
classifications of location type is given in 
appendix A. 

The performance of students by type of 
school location is shown in table 3.1. At 
grades 4 and 8, students attending schools 
in rural and urban fringe locations had 
higher average scores than students in 
central city schools. At grade 12, students 
attending schools in urban fringe locations 
had higher scores than students in both 
rural and central city locations. 




Average U.S. history scale scores by type of location, grades 4, 8, and 12: 2001 



Central city Urban fringe/large town Rural/small town 



Grade 4 


i . "SC ' ; 








: 199 


211 
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Grade 8 
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Grade 12 


283 


292 


284 



SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 2001 U.S. History Assessment. 



Figure 3.13 presents achievement-level 
results by type of school location. Com- 
parisons of achievement-level results 
between locations show higher percentages 
of fourth- and eighth-grade students at or 
above Basic in rural and urban fringe 
locations than in central city locations. 
There was also a higher percentage of 



fourth-graders at or above the Basic level in 
rural schools than in urban fringe schools. 
At grade 12, the percentages of students at 
or above the Basic and Proficient levels were 
higher in schools located in urban fringe 
areas than those in both rural and central 
city locations. 
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ft Percentage is between 0.0 and 0.5. 

NOTE: Percentages within each U.S. history achievement-level range may not add to 100, or to the exact percentages at or above achievement levels, due to rounding. 
SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 2001 U.S. History Assessment. 
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Free/Reduced-Price School 
Lunch Program Eligibility 

Funded by the U.S. Department of Agri- 
culture (USDA) as part of the National 
School Lunch Program, the Free/Re- 
duced-Price School Lunch program is 
designed to assure that children at or near 
the poverty line receive nourishing meals. 
Eligibility guidelines for the lunch program 
are based on the Federal income poverty 
guidelines and are stated by household 
size. 5 NAEP first began collecting data on 
student eligibility for this program in 1996; 
therefore cross-year comparisons back to 
1994 are not possible. Table 3.2 presents 
the 2001 U. S. history scale score results by 
students’ eligibility for the program. At 



every grade, the average scale scores for 
students who were not eligible for the 
Free/Reduced-Price School Lunch pro- 
gram (i.e., those above the poverty guide- 
lines) were higher than the scores for the 
students who are eligible for the program. 
Since information on eligibility is not 
available for a substantial percentage of the 
students at each grade, table 3.2 also dis- 
plays the scale score averages for this third 
group of students. (Some schools do not 
offer free/reduced-price lunches. Students 
from these schools are counted in the 
Information Not Available category.) This 
group also had higher scale scores at every 
grade than the students eligible for the Free/ 
Reduced-Price School Lunch program. 




Average U.S. history scale scores by student eligibility for Free/Reduced-Price School Lunch 
program, grades 4, 8, and 12: 2001 



Eligible Not eligible Info not available 
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SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics. National Assessment of 
Educational Progress (NAEP), 2001 U.S. History Assessment. 



The pattern for achievement-level results 
is displayed in figure 3.14 and parallels that 
seen in the scale scores. There were higher 
percentages of fourth-, eighth, and twelfth- 



grade students at or above Basic and Profi- 
cient who were not eligible for the program 
than those who were eligible for it. 



5 U.S. General Services Administration. (1999). Catalogue of federal domestic assistance. Washington, DC: Executive 
Office of the President, Office of Management and Budget. 
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NOTE: Percentages within each U.S. history achievement-level range may not add to 100, or to the exact percentages at or above achievement levels, due to rounding. 
SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 2001 U.S. History Assessment. 
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in the nation, NAEP consistently has striven to include 
special-needs students — those with disabilities (SD) or 
limited English proficient students (LEP). A certain 
percentage of such students, however, has always been 
excluded because they could not be assessed meaningfully 
without accommodations. Schools that participate in NAEP 
have been asked to use specific criteria in making 
decisions to exclude certain students who have been 
classified as having a disability under the Individuals 
with Disabilities Education Act (IDEA), based upon 
their Individualized Education Programs (IEP) and 
Section 504 of the Rehabilitation Act of 1973. 
Similarly, schools have been permitted to exclude 
some students they identify as being limited English 
proficient. 1 

In order to increase the inclusiveness of 
NAEP’s samples, and in an attempt to remain 
consistent with state- and district-level testing 
policies that increasingly offer accommodations to 
special-needs students, NAEP began to explore the 
use of accommodations in the 1996 and 1998 assessments. A 
split-sample design was used to identify a portion of schools 
that were permitted to provide accommodations to their 
special-needs students who required them, and a portion of 
schools in which accommodations were not offered (the 
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1 See appendix A for a description of specific criteria provided to assist them in making 
exclusion decisions. 
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standard administration procedure prior to 
1996). The split-sample design made it 
possible to study the effects on NAEP 
results of including special-needs students 
who required and were provided accom- 
modations, while at the same time, obtain- 
ing results that were comparable to those 
from previous assessments. Based on re- 
search conducted and published since that 
time, it was determined that NAEP could 
begin a transition to reporting results that 
included the performance of accommo- 
dated special-needs students. 2 It is antici- 
pated that in the near future, NAEP will 
only report results based on this more 
inclusive sample. 

Two Sets of 2001 NAEP 
U.S. History Results 

This report is the first to display two 
different sets of NAEP U.S. history results 
based on the split-sample design: 1) those 
that reflect the performance of regular and 
special-needs students when accommoda- 
tions were not permitted, and 2) those that 
reflect the performance of regular and 
special-needs students — both those who 
were accommodated and those who could 
be tested without accommodations — when 
accommodations were permitted. It should 
be noted that accommodated students 
make up a small proportion of the total 



weighted number of students assessed (see 
table A. 6, page 115 in appendix A for 
details). Making accommodations available 
may change the overall assessment results in 
subtle and different ways. For example, 
when accommodations are permitted, there 
may be some occurrences of students being 
accommodated who might have taken the 
test under standard conditions if accommo- 
dations were not permitted. This could 
lead to an overall increase in the average 
assessment results if accommodations were 
to increase special-needs students’ perfor- 
mance. Conversely, when accommodations 
are permitted, special-needs students who 
could not have been tested without ac- 
commodations could be included in the 
sample. Assuming that these are generally 
lower-performing students, their inclusion 
in the sample — even with accommoda- 
tions — could result in an overall lower 
average score. 

The two sets of results presented in this 
chapter were obtained by administering the 
assessment to a nationally representative 
sample of students and schools. In one 
sample, no accommodations were permit- 
ted; all students were assessed under the 
same conditions that were the basis for 
reporting results from the 1994 NAEP U.S. 
history assessment. In another part of the 



2 Olson, J. F. & Goldstein, A. A. (1997). The inclusion of students with disabilities and limited -English-proficient students in 
large-scale assessments: A summary of recent progress. (NCES Publication No. 97-482). Washington, DC: National 
Center for Education Statistics. 

Mazzeo,J., Carlson, J. E.,Voelkl, K. E., & Lutkus, A. D. (1999). Increasing the participation of special needs students in 
NAEP: A report on 1996 research activities. (NCES Publication No. 2000-473). Washington, DC: National Center 
for Education Statistics. 
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schools sampled, accommodations were 
permitted for students with disabilities and 
limited English proficient students who 
normally receive accommodations in their 
district or state assessment programs. Most 
accommodations that schools routinely 
provide for their own testing programs 
were permitted. The permitted accommo- 
dations included, but were not limited to 
the following: 

□ one-on-one testing, 

□ bilingual dictionary, 

□ large print book, 

□ small-group testing, 

□ extended time, 

□ oral reading of questions, and 

D use of an aide for transcribing responses. 
(See appendix A, table A. 7, page 1 17, for 
greater detail on the numbers and percent- 
ages of students accommodated by accom- 
modation type in the 2001 assessment.) 

Figure 4.1 provides a visual representa- 
tion of how the two sets of results were 
based on the two samples in 2001. In- 
cluded in both sets of results (accommoda- 
tions not permitted and accommodations 
permitted) are those students from both 
samples of schools who were not identified 
as either SD or LEP. In addition, the first 
set of results (accommodations not permit- 
ted) includes SD and LEP students from 
the sample of schools where accommoda- 
tions were not permitted (see middle 
portion of figure 4.1). This is the set of 



0 




results that allows for trend comparisons 
back to 1994 and are presented in the 
other chapters of this report. 

The second set of results, accommodations 
permitted (see bottom portion of figure 
4.1), includes SD and LEP students from 
the sample of schools where accommoda- 
tions were permitted. This is the set of 
results that form the new, more inclusive 
baseline for future reporting of trend 
comparisons for the NAEP U.S. history 
assessment. 

In the NAEP 2001 sample where 
accommodations were not permitted, 

16 percent of fourth-graders, 16 percent of 
eighth-graders, and 1 1 percent of twelfth- 
graders, were identified by their schools as 
having special needs (i.e., either as students 
with disabilities or limited English profi- 
cient students). In the other sample where 
accommodations were offered, 18 percent 
of fourth-graders, 17 percent of eighth- 
graders, and 10 percent of twelfth-graders 
were identified as having special needs. In 
the sample where accommodations were 
not permitted, between 45 and 51 percent 
of the special-needs students at each of the 
three grade levels (between 4 and 8 percent 
of all students — see appendix A, table A. 5, 
page 114) were excluded from NAEP 
testing by their schools. In the sample 
where accommodations were offered, 
between 23 and 33 percent of the special- 
needs students were excluded from the 
assessment (between 2 and 3 percent of the 
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The two sets of NAEP results based on a split-sample design 
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Split-sample design 

The national sample was split. In part of the 
schools, accommodations were not permitted 
for students with disabilities (SD) and limited 
English proficient (LEP) students. In the other 
schools, accommodations were permitted for 
SD and LEP students who routinely received 
them in their school assessments. 



Accommodations-not-permitted results 

The accommodations-not-permitted results 
include the performance of students from both 
samples who were not classified as SD or LEP 
and the performance of SD and LEP students 
from the sample in which no accommodations 
were permitted. 



Accommodations-permitted results 

The accommodations-permitted results also 
include the performance of students from both 
samples who were not classified as SD or LEP; 
however, the SD and LEP students whose 
performance is included in this set of 
results were from the sample in which 
accommodations were permitted. Since 
students who required testing accommodations 
could be assessed and represented in the 
overall results, it was anticipated that these 
results would include more special-needs 
students and reflect a more inclusive sample. 
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total sample) . Thus, offering accommoda- 
tions would appear to lead to greater 
inclusion of special-needs students. 

Chapters 2, 3, 5, and 6 of this report are 
based on the first set of results (no accom- 
modations permitted). This chapter pre- 
sents an overview of the second set of 
results — results that include students who 
were provided accommodations during the 
assessment administration. Overall results 
are provided for the nation and for student 
subgroups by gender and by race/ethnicity. 
These results are discussed in terms of 
statistically significant differences between 
the two sets of results and differences 
between subgroups of students within each 
set of results. Throughout this chapter, the 
assessment results that include SD and LEP 
students for whom accommodations were 
not permitted will be referred to as the 
“accommodations-not-permitted” results. 
The set of results that includes SD and LEP 
students for whom accommodations were 
permitted will be referred to as the “ac- 
commodations-permitted” results. 



0 




Results for the Watson 

Accommodations Not Permitted and 
Accommodations Permitted 

Table 4.1 displays the average U.S. history 
scale scores for the nation in 2001 for two 
sets of results: 1) accommodations not 
permitted, and 2) accommodations permit- 
ted. There were no significant differences 
in the average scores between the two sets 
of results at grades 4 and 12. At grade 8, 
however, the average score when accom- 
modations were permitted was lower than 
the average score when accommodations 
were not permitted. 

As noted in the introduction to this 
chapter, NAEP has always sought to in- 
clude special-needs students proportional 
to their representation in the U.S. popula- 
tion. Offering accommodations tends to 
reduce exclusion rates for special-needs 
students and therefore allows NAEP to 
offer a fairer and more accurate picture of 
the status of American education. Because 
special-needs students are typically classi- 
fied as eligible for special educational 




CHAPTER 4 



U.S 



HISTORY REPORT CARD 



53 





National average U.S. history scale 


scores by type of results, grades 4, 8, and 12: 2001 




Accommodations not permitted 


Accommodations permitted 


Grade 4 


j 

209 


208 


Grade 8 


262 


260 1 


Grade 12 


287 


287 



t Significantly different from the sample where accommodations were not permitted. 

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 2001 U.S. History Assessment. 



services after having shown some difficulty 
in the regular learning environment, some 
may assume that including the perfor- 
mance of these students would tend to 
lower the overall results. This assumption 
appears to have been justified only in the 
observed difference between the two sets 
of grade 8 U. S. history results in 2001, 
where the accommodations-permitted 
results, which included slightly more 
special-needs students because of the 
availability of accommodations, were lower 
than the accommodations-not-permitted 
results. It is important to examine the 
percentages of students attaining the NAEP 
achievement levels, however, to see if there 



were higher percentages at the lower 
achievement level (i.e., Basic), when stu- 
dents were assessed with accommodations. 

Table 4.2 shows the percentages of 
students attaining each of the achievement 
levels. The percentages are similar across 
the two sets of results for grades 4 and 12; 
apparent differences between the 
accommodations-not-permitted and the 
accommodations-permitted results were 
not significantly different. At grade 8, 
however, the percentage of students below 
Basic was higher when accommodations 
were permitted than when they were not 
permitted. 
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Percentage of students within and at or above U.S. history achievement levels by type of results, 
grades 4, 8, and 12: 2001 









r _ 


















At or above 


At or above 




Below Basic f 


At Basic 


At Proficient 


At Advanced 


Basic 


Proficient 


Grade 4 

Accommodations were not permitted 


33 


49 


16 


2 


67 


18 


Accommodations were permitted 


1 34 


48 


16 


2 


66 


18 


Grade 8 

Accommodations were not permitted 


36 


48 


15 


, 2 


64 


17 


Accommodations were permitted 


38 * 


46 ♦ 


14 


1 


62 t 


16 


Grade 12 

Accommodations were not permitted 


5? 


32 


10 


1 ! 


43 


11 


Accommodations were permitted 


57 


32 


10 

i 


1 


43 


11 



t Significantly different from the sample where accommodations were not permitted. 

NOTE: Percentages within each U.S. history achievement-level range may not add to 100, or to the exact percentages at or above achievement levels, due to rounding. 
SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 2001 U.S. History Assessment. 



National Results by Gender 

Accommodations Not Permitted and 
Accommodations Permitted 

The average U.S. history scale scores by 

gender for both sets of results in 2001 are 

provided in table 4.3. Both male and 

female students at grade 8 had higher U.S. 

history scores when accommodations were 



not permitted than when accommodations 
were permitted. 

At all three grades, the average scores for 
male students were not significantly differ- 
ent from that of female students regardless 
of whether or not accommodations were 
permitted. 




National average U.S. history scale scores by gender and type of results, grades 4, 8, and 12: 2001 





Male 


Female 


Grade 4 




! 


Accommodations were not permitted 


209 


1 209 


Accommodations were permitted 


207 


209 


Grade 8 






Accommodations were not permitted i 


264 


261 


Accommodations were permitted ; 


261 1 


260 t 


Grade 12 






Accommodations were not permitted 


288 


286 


Accommodations were permitted \ 


288 


286 


t Significantly different from the sample where accommodations were not permitted. 



SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 2001 U.S. History Assessment. 
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The percentages of male and female 
students attaining the Basic , Proficient, and 
Advanced levels are provided in table 4.4. 
Comparing the two sets of results in 2001, 
there were no statistically significant differ- 
ences in the percentages of male or female 



students attaining each of the achievement 
levels at grades 4 or 12. At grade 8, how- 
ever, a higher percentage of male students 
were below Basic when accommodations 
were permitted than when they were not. 




Percentage of students within and at or above U.S. history achievement levels by gender and type 
of results, grades 4, 8, and 12: 2001 













At or above 


At or above 




Below Basic 


At Basic 


At Proficient j 


At Advanced 


Basic 


Proficient j 


Grade 4 
Male 

Accommodations were not permitted 


34 


47 


! ! 

17 


2 


66 


19 


Accommodations were permitted 


35 


46 


16 


2 


65 


19 


Female 

Accommodations were not permitted 


32 


5! 


: 1 

15 


2 


68 


17 


Accommodations were permitted 


33 


50 


15 


2 


67 


17 


Grade 8 
Male 

Accommodations were not permitted 


35 


47 


1 j 

m i7 i 


2 


65 


18 


Accommodations were permitted 


38* 


45 


16 


2 


62 1 


17 


Female 

Accommodations were not permitted 


37 


48 


14 


i i 


63 


15 


Accommodations were permitted 


39 


47 


13 


i 


61 


14 


Grade 12 
Male 

Accommodations were not permitted 


55 


33 


1 

ii 


i 


45 


12 


Accommodations were permitted 


55 


32 


! ii 


2 


45 


12 


Female 

Accommodations were not permitted 


59 


31 


9 


1 


41 


10 


Accommodations were permitted 


60 


31 


j 9 


1 


40 


10 



1 Significantly different from the sample where accommodations were not permitted. 

NOTE: Percentages within each U.S. history achievement-level range may not add to 100, or to the exact percentages at or above achievement levels, due to rounding. 
SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 2001 U.S. History Assessment. 
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National Results by 
Race/Ethnicity 

Accommodations Not Permitted and 
Accommodations Permitted 

NAEP assessments across academic subjects 
have typically reported large score differ- 
ences according to race and ethnic group 
membership. If students with disabilities or 
limited English proficient students are 
over-represented in a particular racial or 
ethnic group, that group s assessment scores 
may decrease. Table 4.5 provides the 
average U. S. history scale scores for each of 
the race/ethnicity categories for the two 
sets of results in 2001. At grade 8, both 
White students and Black students had 
higher average scores when accommoda- 
tions were not permitted than when 
accommodations were permitted. There 
were no statistically significant differences 



observed between the average scores when 
accommodations were not permitted and 
when accommodations were permitted for 
any of the race/ethnicity categories at 
grades 4 and 12. 

As noted in chapter 3, a pattern of 
performance differences by race/ethnicity 
can be seen in the accommodations-not- 
permitted results in 2001. Both White and 
Asian/Pacific Islander students at all three 
grades scored higher than Black and 
Hispanic students. The same pattern can be 
observed in the accommodations-permit- 
ted results. However, while White students 
outperformed their Asian/Pacific Islander 
peers at grade 4 when accommodations 
were not permitted, the difference was not 
statistically significant when accommoda- 
tions were permitted. 




National average U.S. history scale scores by race/ethnicity and type of results, grades 4, 8, and 12: 
2001 



Asian/Pacific American 

White Black Hispanic Islander Indian 



Grade 4 












Accommodations were not permitted 


220 


188 


186 


213 


197 


Accommodations were permitted 


218 


186 


187 


214 


197 


Grade 8 

Accommodations were not permitted 


1 271 


243 


243 


267 


249 


Accommodations were permitted 


269 t 


240 r 


240 


265 


248 


Grade 12 

Accommodations were not permitted j 


292 


269 


274 


295 


277 


Accommodations were permitted \ 

1 


292 


268 


271 


294 


274 



1 Significantly different from the sample where accommodations were not permitted. 

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 2001 U.S. History Assessment. 



The percentages of students in each 
race/ethnicity category who attained the 
Basic, Proficient , and Advanced levels are 
provided in table 4.6. No significant differ- 
ences were found at any of the three grades 



between the accommodations-not-permit- 
ted results and the accommodations- 
permitted results for the percentages of 
students attaining each of the achievement 
levels in 2001. 
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Percentage of students within and at or above U.S. history achievement levels by race/ethnicity 
and type of results, grades 4, 8, and 12: 2001 







; 














r~ 






At or above 


; At or above 




Below Basic 


At Basic 


At Proficient 


At Advanced 


Basic 


Proficient 


Grade 4 
White 

Accommodations were not permitted 


21 


\ 

55 


\ i 

i 1 

21 ; 


3 


79 


24 


Accommodations were permitted 


23 


53 


I 21 


3 


77 


24 


Black 

Accommodations were not permitted 


56 


38 


| ! 
i 5 i 


# 


44 


6 


Accommodations were permitted 


58 


36 


i 5 


# 


42 


5 


Hispanic 

Accommodations were not permitted 


58 


35 


6 


1 


42 


7 


Accommodations were permitted 


58 


36 


! 6 


# 


42 


6 


Asian/Pacific Islander 

Accommodations were not permitted 


29 


53 


| 16 


3 


71 


19 


Accommodations were permitted 


26 


54 


17 


3 


74 


20 


American Indian 

Accommodations were not permitted 


47 * 


41 


8 ! 


4 


53 


12 


Accommodations were permitted 


44 


44 


9 


3 


56 


12 


Grade 8 
White 

Accommodations were not permitted 


25 


53 


i j 

19 


2 


75 


21 


Accommodations were permitted 


27 


52 


19 ! 


2 


73 


20 


Black 

Accommodations were not permitted 


62 


34 


4 


# 


38 


4 


Accommodations were permitted 


65 


31 


4 


# 


35 


4 


Hispanic 

Accommodations were not permitted 


60 « 


- 34 


5 


# 


40 


5 


Accommodations were permitted 


63 


32 


4 


# 


37 


4 


Asian/Pacific Islander 
Accommodations were not permitted 


32 *• 


48 ‘ 


18 


2 


68 


20 


Accommodations were permitted 


34 


47 


17 


2 


66 


19 


American Indian 

Accommodations were not permitted 


50 


42 


7 


1 


50 


8 


Accommodations were permitted 


54 


38 


S 7 


1 


46 


8 


Grade 12 
White 

Accommodations were not permitted 


51 


36 


l 

12 


1 


49 


13 


Accommodations were permitted 


51 


36 


12 


1 


49 


13 


Black 

Accommodations were not permitted 


80 


18 


;} 

! 3 


# 


20 


3 


Accommodations were permitted 


80 


17 


! 3 


# 


20 


3 


Hispanic 

Accommodations were not permitted 


74 


21 


1 5 


# 


26 


5 


Accommodations were permitted 


74 


21 


5 


# 


26 


5 


Asian/Pacific islander 
Accommodations were not permitted 


47 


31 


17 


5 


53 


21 


Accommodations were permitted 


48 


31 


16 


5 


52 


21 


American Indian 

Accommodations were not permitted 


66 


33 


1 


0 


34 


1 


Accommodations were permitted 


68 


31 


1 


0 


32 


1 



# Percentage is between 0.0 and 0.5. 

NOTE: Percentages within each U.S. history achievement-level range may not add to 100, or to the exact percentages at or above achievement levels, due to rounding. 
SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 2001 U.S. History Assessment. 
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Sample Assessment Questions 
and Student Responses 



This chapter presents sample questions from the 2001 
NAEP U.S. history assessment and examples of student 
responses to those questions. Four sample questions at each 
grade level are provided, including multiple-choice, short 
constructed-response, and extended constructed-response 
questions. Each sample question is classified according to a 
historical theme or historical period, as described in the U.S. 
history framework. Actual student responses to the 
sample constructed-response questions have been 
reproduced from test booklets to illustrate answers 
representative of the indicated scoring categories. 

The tables accompanying each sample question 
present two types of performance data: the overall 
percentage of students who answered successfully and 
the percentage of students within a specific score 
range on the NAEP U.S. history scale who answered 
successfully. The score ranges presented are those 
that correspond to the three achievement-level 
intervals — Basic, Proficient, and Advanced — as well as 
the score range that falls below Basic. 




Materials from 
the 2001 
U.S. history 
assessment 




Sample 

Questions 

Studeii 

Responses 

Item Maps 
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Grade 4 Sample Assessment 
Question Results 

Assessment questions at grade 4, as at the 
other grade levels, included both con- 
structed-response and multiple-choice 



formats. Many, like the examples below, 
used visual or textual stimuli. Questions 
tended to concern topics that are typically 
addressed in fourth-grade social studies 
classes. 



(Oi)©saa®GD t)g 




In pioneer schools, feathers like this were most often used for 
® measuring 
<D sewing 
O writing 
<E> playing a game 





[MailMIIMMls 


Economic and Technological Changes 


Expansion and Reform 


and Their Relation to Society, Ideas, and 


(1801 to 1861) 


the Environment 




TMfeO© a,tl SaGuDQfi© ODDS^flCilD 11 



Overall percentage correct and percentages correct within each achievement level range: 2001 








Overall percentage 


Below Basic j 


Basic 


S Proficient 


Advanced 


correct 


194 and below* 


195-242* 


243-275* 


276 and above* 


93 


84 j 


96 

! 


j 99 . 


*** 



*NAEP U.S, history composite scale range. 

***Samp!e size insufficient to permit a reliable estimate (see appendix A). 

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National 
Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2001 U.S. History 
Assessment. 
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FOR U S. ARMY 

NEAREST RECRUITING STATION 



The poster shown above is trying to attract recruits by appealing to their 
CD homesickness 
CD religious beliefs 
O patriotism 
CD need for money 




The Changing Role of America in the World Modern America and the World Wars 

(1914 to 1945) 




Overall percentage correct and percentages correct within each achievement level range: 2001 





MteMueaWM tflftsoisate 


Overall percentage 


Below Basic 


Basic 


Proficient 


Advanced 


correct 


194 and below* 


195-242* 


243-275* 


276 and above* 


45 


25 


46 


i 

1 74 

! 

i 


*** 




*NAEP U.S. history composite scale range. 

***Sample size insufficient to permit a reliable estimate (see appendix A). 

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National 
Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2001 U.S. History 
Assessment. 
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What was a major cause of the Civil War? 

d> People in the North and in the South had different religions. 

O People in the North and in the South disagreed over slavery. 

CD People in the North wanted control of the country when they 
found out that gold had been discovered in the South. 

CD People in the South wanted control of the country when they 
found out that oil had been discovered in the North. 




Change and Continuity in American Democracy: 
Ideas, Institutions, Practices, and Controversies 



Crisis of the Union: Civil War and Reconstruction 
(1850 to 1877) 




Overall percentage correct and percentages correct within each achievement level range: 2001 







Overall percentage 
correct 


Below Basic 
194 and below* 


Basic 

195-242* 


Proficient 

243-275* 


Advanced 
276 and above* 


57 


34 


59 


86 

1 . . 


*** 



*NAEP U.S. history composite scale range. 

***Sample size insufficient to permit a reliable estimate (see appendix A). 

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National 
Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2001 U.S. History 
Assessment. 
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Choose an American Indian group from the map, and circle its name directly 
on the map. 

On the chart below, list one way this American Indian group got food, 
shelter, and clothing in the period before Europeans came to the 
Americas. Then list one way your family gets food, shelter, and clothing. 





American Indians in the Period 
Before Europeans Came 




Your Family 


1. 


Food: 


1. 


Food: 










2. 


Shelter: 


2. 


Shelter: 










3. 


Clothing: 


3. 


Clothing: 



Give one reason why the American Indian group long ago and your family 
today differ in the ways they get their food, shelter, or clothing. 




The Gathering and Interactions of Peoples, Three Worlds and Their Meeting in the Americas 

Cultures, and Ideas (Beginnings to 1607) 
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Responses to this question were scored according to a four-level rubric as 
Inappropriate, Partial, Essential, or Complete. 




Overall percentage “Essential” or better and percentages “Essential” or better within each achievement 
level range: 2001 





!Mr®M£p “118930^ 


Overall percentage 


Below Basic 


Basic 


Proficient Advanced 


“Essentia!” or better 


194 and below* 


195-242* 


243-275* 276 and above* 


42 


13 


48 


76 *** 



*NAEP U.S. history composite scale range. 

***Sample size insufficient to permit a reliable estimate (see appendix A). 

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National 
Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2001 U.S. History 
Assessment. 
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Responses scored “Complete” circled an American Indian group and correctly listed ways 
that the group chosen got food, shelter, and clothing, and ways that the student’s own family 
gets food, shelter, and clothing. They then gave one appropriate reason for differences 
between the way the American Indian group obtained those necessities and the way in which 
modern families obtain them. 

Sample “Complete” Response: 




American Indians in the Period 
Before Europeans Came 



1. Food: 



iPof-H p\ A 

s r 



Your Family 

L Food: We ao jo Hv 



2. Shelter: 



tYlA.rice^'t 

Wp Uu.i^ ^iq'isWA 



'oonlW* houses. 

3. Clothing: We Worr^ 



sW 



2. Shelter: ~TVecj loo'i\ Ho^SeS 

yrraAe \re<zS 

3. Clothing: VV^e 

S kl h ot" o^'n'ima^S 

V.'UecL 

ason why the i 

today differ in the ways they get their food, shelter, or clothing. 

Wg. Wcflasg. Ct \\m£ 

t\ao ipererO-V qnq stores or reoA 

e-sVftcVe people c^s Wave, ~ 



roa - 

Give one reason why the American Indian group long ago and your family 



mi: 
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Responses scored “Essential" circled an American Indian group and correctly listed two ways 
that the group chosen got food, shelter, or clothing, and two ways that modern families obtain 
them. They did not give an appropriate reason for these differences. 

Sample “Essential” Response: 




American Indians in the Period 
Before Europeans Came 

1. Food: f/uLlHleA 


Your Family 

1. Food: G?/0CeY^ S +C 


2. Shelter: ~tY\£M IASGcX 


2. Shelter: 


'"Ro-f? -al o K*de 


3. Clothing: /Lc l Aifc 


3. Clothing: JJ^p af-VwOlt" 


Shoves 



Give one reason why the American Indian group long ago and your family 
today differ in the ways they get their food, shelter, or clothing. 



We — — bxLa — fh i A^S 
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Responses scored “Partial" circled an American Indian group and were able to identify 
correctly one way the group chosen got food, shelter, or clothing and one way modern families 
get them. 

Sample “Partial” Response: 




American Indians in the Period 
Before Europeans Came 



1. Food: 



2. Shelter: 



3. Clothing: 



uT 



1. Food: 



Your Family 



2 - shelter: 

^o- 

3. Clothing: 



rr^CxAjl 

Give one reason why the American Indian group long ago and your family 
today differ in the ways they get their food, shelter, or clothing. . 

si2svzs€>( 

'dpy-' • 
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Grade 8 Sample Assessment 
Question Results 

In addition to assessing content appropriate 
to an eighth-grade U.S. history course, 



assessment questions for eighth grade 
assessed a range of history skills, such as the 
text interpretation skills measured in sample 
question 8. 




Why was Roger Williams forced to leave the Massachusetts Bay Colony? 

O He claimed that the Puritan government had no right to control 
religious beliefs. 

CD He was more loyal to the King of Spain than to the English 
monarchy. 

(D He refused to do his share of the farming and other work. 

CD He wanted to lead a war against the American Indians. 




Change and Continuity in American Democracy: Colonization, Settlement, and Communities 

Ideas, Institutions, Practices, and Controversies (1607 to 1763) 




Overall percentage correct and percentages correct within each achievement level range: 2001 





g3@&ft3W30u)®][C<d)C^^ 


Overall percentage 


Below Basic i 


Basic 


Proficient 


Advanced 


correct 


251 ami below* 


252-293* 


294-326* 


327 and above* 


52 


34 ! 


56 


79 

i 


*** 



*NAEP U.S. history composite scale range. 

***Sample size insufficient to permit a reliable estimate (see appendix A). 

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National 
Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2001 U.S. History 
Assessment. 
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What was the most significant factor that led the American colonists to form 
the First Continental Congress in 1774 ? 

CD Religious conflict inside the colonies 

CD The desire of the colonists to write a Constitution to replace the 
Articles of Confederation 

O Colonial frustration with laws passed by the British Parliament 

CD The desire of the colonists to stop the war between Britain and 
the colonies 




Change and Continuity in American Democracy: The Revolution and the New Nation 

Ideas, Institutions, Practices, and Controversies (1763 to 1815) 




Overall percentage correct and percentages correct within each achievement level range: 2001 





— - . — . - . — i 


Overall percentage 


Below Basic 


Basic 


Proficient 


Advanced 


correct 


251 and below* 


252-293* | 


; 294-326* 


327 and above* 


.v 39 - 


29 


39 


62 


*** 



*NAEP U.S. history composite scale range. 

***Sample size insufficient to permit a reliable estimate (see appendix A). 

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National 
Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2001 U.S. History 
Assessment. 
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Why was the invention of the steel plow important in United States 
history? 




Economic and Technological Changes and Their The Development of Modern America 
Relation to Society, Ideas, and the Environment (1805 to 1920) 



Responses to this question were scored according to a three-level rubric as 
Inappropriate, Partial, or Appropriate 




Overall percentage “ Appropriate” and percentages “Appropriate” within each achievement level 
range: 2001 



Me® 




Overall percentage 
“Appropriate” 


Below Basic 
251 and below* 


Basic 

j 252-293* 


Proficient Advanced 

294-326* 327 and above* 


30 

■ ■ ; y- 


9 


34 

1 


64 *** 



*NAEP U.S. history composite scale range. 

***Sample size insufficient to permit a reliable estimate (see appendix A). 

SOURCE*. U.S. Department of Education, Office of Educational Research and Improvement, National 
Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2001 U.S. History 
Assessment. 
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This “Appropriate” response indicated that the steel plow increased efficiency in agricultural 
production. 



Sample “Appropriate” Response: 



Why was the invention of the steel plow important in United States 
history? 



Ik sW\ u;ah> \aA &A uJcw-Wdl ksW , anA CoAi 

VxarWr c^o-u/aX . 



This "Partial” response indicated in a general way that the steel plow made farming easier; it 
was correct but was not specific about the steel plow’s impact. 



Sample “Partial” Response: 

Why was the invention of the steel plow important in United States 
history? 

maKi cl \oV ■ 
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In America, it was no disgrace to work at a trade. Workmen and 
capitalists were equal. The employer addressed the employee as you, 
not familiarly as thou. The cobbler and the teacher had the same title, 
"mister," and all the children, boys and girls, Jews and Gentiles, went 
to school! 

— Polish immigrant, 1910 



Using the quotation above and your knowledge of history, explain in your 
own words two important aspects of life in the United States that seemed 
good to this immigrant. 

1 ) 



2 ) 



What do you think was the most important difference this man saw 
between Poland and the United States? 






The Gathering and Interactions of Peoples, 
Cultures, and Ideas 



The Development of Modern America 
(1865 to 1920) 
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Responses to this question were scored according to a three-level rubric as 
Inappropriate, Partial, or Appropriate 




Overall percentage “Appropriate” and percentages “Appropriate” within each achievement level 
range: 2001 



Me® 




rr Overall percentage 


i Below Basic 


1 Basic 


1 Proficient Advanced 


"Appropriate" 


j 251 and below* 


1 252-293* 


294-326* 327 and above* 


"■V 


i •. ii 

1 


CO £ 
CO 


64 *** 



*NAEP U.S. history composite scale range. 

***Sample size insufficient to permit a reliable estimate (see appendix A). 

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National 
Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2001 U.S. History 
Assessment. 



Responses scored “Appropriate” correctly identified two positive aspects of American life as 
perceived by the immigrant quoted and explained an important distinction between the 
United States and Poland in the eyes of that immigrant. 



Sample “Appropriate” Response: 



Using the quotation above and your knowledge of history, explain in your 
own words two important aspects of life in the United States that seemed 



good to this immigrant. 

V TjL 






O 









JUjl 









J^CTL> 










What do you think was the most important difference this man saw 
between Poland and the United States? 






A-gi/iT' 
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Responses scored “Partial" correctly identified two positive aspects of American life as 
perceived by the immigrant quoted but did not explain an important distinction between the 
United States and Poland in the eyes of the immigrant. 



Sample “Partial” Response: 



Using the quotation above and your knowledge of history, explain in your 
own words two important aspects of life in the United States that seemed 
good to this immigrant. 









&A 








What do you think was the most important difference this man saw 
between Poland and the United States? 









/O' 






/Qcfy^sL, 
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Grade 12 Sample Assessment 
Question Results 

Questions at the twelfth-grade level 
assessed U.S. history knowledge and skills 
at a more sophisticated level than those at 
grades 4 and 8. This greater complexity is 



presented in the questions included below 
both in the degree of detail required and 
the extent to which students must under- 
stand and analyze historical issues from 
various perspectives, as is evident in the 
constructed-response questions. 




The Progressive movement of 1890-1920 is best described as 

O a broad-based reform movement that tried to reduce the abuses 
that had come with modernization and industrialization 

CD a loose coalition of groups primarily dedicated to passing a consti- 
tutional amendment prohibiting the consumption of alcohol 

CD an anti-tariff movement led by a federation of business owners 
and manufacturers who wanted to promote trade abroad 

<E> a grass-roots movement that attempted to gather support for the 
establishment of an international organization such as the League 
of Nations 







Change and Continuity in American Democracy: 
Ideas, Institutions, Practices, and Controversies 


The Development of Modern America 
(1865 to 1920) 


TMfeO© ggmolte ®DD©@Qftm) © Bssooffe ®5)QD0QQijfl©=<^^ 



Overall percentage correct and percentages correct within each achievement level range: 2001 



Mfefjg 




Overall percentage 


Below Basic 


Basic 


Proficient 


Advanced 


correct 


293 and below* 


294-324* 

i 


325-354* 


355 and above* 


36 


23 


47 !. 


73 


*** 



*NAEP U.S. history composite scale range. 

***Sample size insufficient to permit a reliable estimate (see appendix A). 

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National 
Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2001 U.S. History 
Assessment. 
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The phrase "Harlem Renaissance" refers to 

<3> African American political gains during the Reconstruction 
period 

O African American achievements in art, literature, and music in 
the 1920's 

<D a religious revival in the African American community that 
swept the nation in the 1950's 

® a series of urban renewal projects that were part of the Great 
Society program of the 1960's 









The Gathering and Interactions of Peoples, Modern America and the World Wars 

Cultures, and Ideas (1914 to 1945) 




Overall percentage correct and percentages correct within each achievement level range: 2001 



SoadbUS 






i% : 


■ Overall percentage 


Below Basic 


Basic 


i Proficient 


Advanced 


correct 


293 and below* 1 


294-324* 


325-354* 

1 


355 and above* 


<7> 

CO 

isr 


54 < | 


82 


| 95 


*** 



*NAEP U.S. history composite scale range. 

***Sample size insufficient to permit a reliable estimate (see appendix A). 

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National 
Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2001 U.S. History 
Assessment. 
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Article I, Section 2, United States Constitution 

"[The population of the states] shall be determined by adding to the 
whole number of free persons . . . three-fifths of all other persons." 



An important debate led to the writing of this section of the 
Constitution. Identify the issue being debated. 



Describe the northern position in this debate and explain why 
many northerners took it. 



Describe the southern position in this debate and explain why 
many southerners took it. 






Oa^tetrftSSfl (RsoftiXife 



Change and Continuity in American Democracy: The Revolution and the New Nation 

Ideas, Institutions, Practices, and Controversies (1763 to 1815) 
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Responses to this question were scored according to a four-level rubric as 
Inappropriate, Partial, Essential, or Complete 




Overall percentage “Essential” or better and percentages “Essential” or better within each achievement 
level range: 2001 





QEgODflss^oD^ Ooaaaj^gfte 


Overall percentage 


— ~~ — r 

Below Basic j j 


Basic 


Proficient Advanced 


“Essential” or better 


293 and below* i ; 


294-324* 


325-354* 355 and above* 


21 


4 ! 

1 1 


30 


74 *** 



*NAEP U.S. history composite scale range. 

***Sample size insufficient to permit a reliable estimate (see appendix A). 

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National 
Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2001 U.S. History 
Assessment. 



Responses scored “Complete” identified and explained the debate between the North and the 
South over the counting of slaves for purposes of representation, making explicit the relation- 
ship between counting the slaves and representation in Congress. 



Sample “Complete” Response: 

An important debate led to the writing of this section of the 
Constitution. Identify the issue being debated. 



-iL, 



an 












Lu^l 



Describe the northern position in this debate and explain why 
many northerners took it. 




Describe the southern position in this debate and explain why 
many southerners took it. 



XLj^ b cPUA^j 




l 32 
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Responses scored “Essential” identified the debate without explicitly noting the desire of the 
southern states to maximize their voting power in the House of Representatives. 

Sample “Essential" Response: 

An important debate led to the writing of this section of the 
Constitution. Identify the issue being debated. 

tv 



Describe the northern position in this debate and explain why 
many northerners took it. 



C-^Jx3rU{jd 



Describe the southern position in this debate and explain why 
many southerners took it. 

" TK-0- ptsjJXfrv u lag jo 

Ji^we 
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Responses scored “Partial" identified the issue but did not make the northern and southern 
positions clear. 

Sample “Partial” Response: 



An important debate led to the writing of this section of the 
Constitution. Identify the issue being debated. 






Q . 











Describe the northern position in this debate and explain why 
many northerners took it. 



YYn~y\^A, 



Describe the southern position in this debate and explain why 
many southerners took it. 



T /SQ-Oti^L kSL 



/IVL 









IT'T 






00 £L. 
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"In spite of the obvious advantages held by the North, the South 
was able to fight for four years and to achieve some real military 
successes. So while the North held most of the cards, the South had 
one or two aces up its sleeves." 



Identify two of the "aces" (significant advantages) that the 

South had in the Civil War. Explain how these advantages helped the 

South. 




Change and Continuity in American Democracy: Crisis of the Union: Civil War and Reconstruction 

Ideas, Institutions, Practices, and Controversies (1850 to 1877) 
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Responses to this question were scored according to a four-level rubric as 
Inappropriate, Partial, Essential, or Complete 



IMfe teflteoo t)§ OssaDEfe fidsQsoocited] 

Overall percentage “Essential” or better and percentages “Essential” or better within each achievement 
level range: 2001 





“IMadM 1 ® a? IM&t ®KMoo 


wvm m 






Overall percentage 


Below Basic 


Basic j 


Proficient Advanced 


“Essential” or better 


293 and below* 


294-324* | 


325-354* 355 and above* 


39 


17 


62 


oo 

oo 

* 

* 

* 



*NAEP U.S. history composite scale range. 

***Sample size insufficient to permit a reliable estimate (see appendix A). 

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National 
Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2001 U.S. History 
Assessment. 



Responses scored “Complete" identified two advantages held by the South and explained 
how each advantage identified aided the southern war effort. 

Sample “Complete” Response: 

Identify two of the "aces" (significant advantages) that the 
South had in the Civil War. Explain how these advantages helped the 
South. 

ZAe Sqjj/A A /A 

AllJm S'/ 

•/>?<*/> 

■tt) /'A &S Ose/A &_S_ 

h&jui oj <2. afcsfcALAe 

ihp rPAsh 0 /A<?r- 

<*>f f£&. AAAA 
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Responses scored "Essential” identified one advantage the South had and explained how this 
advantage aided the southern war effort (or the responses identified two advantages but did 
not fully explain how both advantages aided the southern war effort). 



Sample “Essential” Response: 

Identify two of the "aces" (significant advantages) that the 

South had in the Civil War. Explain how these advantages helped the 

South. 



fHosbl-ihg. QVii. Ulor urns flight an so uMwn 



loud, of). 



dfu. -Wo cucas 



msirhoi jh&soulkprn ^soldiers InAuJ Jthp 
- h/ira/n hfc^r-(h(in. ifb , nOdfurn solduyis 
'%.sl Imnd ficus ntts rthad dhe 

£Skil^i£M££S uML asm do44u, uv/(Hmu~ 
toftdiht wr-fhpvMCS gJSJCL nof. , 



Responses scored "Partial” identified one advantage the South had, but did not sufficiently 
explain how this advantage aided the southern war effort. 



Sample “Partial” Response: 



Identify two of the "aces" (significant advantages) that the 

South had in the Civil War. Explain how these advantages helped the 

South. 



PcNQ drjtnv£v cpvwVr ^ - 

i ics Oa 3 v\ CQO i vjj-t/pg- 

r . ' Tv 

caors ~~ 

->V.a>rr\ cnfA—t>^c\ Vx^vml CO^H 
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SWiaps of Selected Stem 
Descriptions on the 
NAEP U.S. History Scale — 
Grades 4, 8, and 12 

The U.S. history performance of fourth-, 
eighth-, and twelfth-graders can be illus- 
trated by maps that position item descrip- 
tions along the NAEP U.S. history scale 
where items are likely to be answered 
successfully by students. 1 The descriptions 
used on the maps focus on the U.S. history 
knowledge or skill needed to answer the 
question. For multiple-choice questions, 
the description indicates the knowledge or 
skill demonstrated by selection of the 
correct option; for constructed-response 
questions, the description takes into ac- 
count the knowledge or skill specified by 
the different levels of scoring criteria for 
that question. Five of the questions de- 
scribed on the item maps are included 
among the sample questions in the preced- 
ing section. Each of these sample questions 
is identified as such on the item map. 

Figures 5.1 through 5.3 are item maps 
for grades 4, 8, and 12, respectively. The 
map location for each question identifies 
where that question was answered success- 
fully by at least 65 percent of the students 
for constructed-response questions and 74 
percent of the students for four-option 
multiple-choice questions. For each ques- 
tion indicated on the map, students whose 
average score fell at or above the scale 
point had a higher probability of success- 
fully answering the question, and students 
whose average score was below that scale 
point had a lower probability of success- 
fully answering the question. 



As an example of how to interpret the 
item maps, consider the multiple-choice 
question in figure 5.1 that maps at score 
point 245. As the description indicates, 
fourth-graders were required to “identify 
how railroad affected Chicago.” As this was 
a four-option multiple-choice question, 
students with an average score at or above 
245 had at least a 74 percent probability of 
answering the question correctly. Students 
with an average score below 245 had less 
than a 74 percent probability of doing so. 
This does not mean that all students with 
an average score of 245 or above always 
answered the question correctly, or that all 
students scoring below 245 always an- 
swered the question incorrectly. Rather, the 
item map indicates higher or lower prob- 
ability of answering the question success- 
fully depending on students’ overall U.S. 
history ability as measured by the NAEP 
scale. 

The three U.S. history achievement 
levels for a specific grade are indicated on 
the item map for that grade. It is important 
to note that, although the same 0—500 U.S. 
history scale is used at each grade, the 
achievement levels are grade specific and 
each achievement level begins at a different 
score point at each grade. 




1 Details on the procedures used to develop item maps are provided in appendix A. 
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Advanced 



2 ! 



• 300 Identify branch of government responsible for passing laws 

295 Recognize immigrant quotation 
291 Describe colonial-era hardships 
288 Describe ways American Indians helped Pilgrims 

284 Recognize excerpt from Declaration of Independence 
282 Explain importance of one of the Founding Fathers 



Proficient 



210 : — 270- Use map to identify continent from which slaves came 
267 Identify purposes of key 1 8th-century document 

263 Compare an American Indian group’s way of life with that of families today—Sample Question 4 
258 Recognize that theme of photographs is racial prejudice 

252 Describe American Indian use of animal resources 
247 Identify factor affecting American Revolution 
245 Identify how railroad affected Chicago 

240- Identify purpose of 1 9th-century means of transportation 
235 Recognize name for anti-segregation laws 
230- Identify reason for major piece of progressive legislation 



224 Identify reason for location of 1 9th-century industry 



2 18 Identify government spending pattern from graph 
216 Identify region for colonial economic activity 

209 Recognize cultural significance of totem poles 



Basic 






| 192 Use historical photograph to make inference about suffragists' goal 
L 191 Identify status of the author of 19th-century letter 



184 Describe way in which American Indians interacted with colonists 



NOTE: Regular type denotes a constructed-response question. Italic type denotes a multiple-choice question. 

* Each grade 4 U. S. history question in the 2001 assessment was mapped onto the NAEP 0-500 U. S. history scale. The position of the question on the scale represents the scale score 
attained by students who had a 65 percent probability of successfully answering a constructed-response question, or a 74 percent probability of correctly answering a four-option 
multiple-choice question. Only selected questions are presented. Scale score ranges for U. S. history achievement levels are referenced on the map. For constructed-response questions, the 
question description represents students' performance at the scoring criteria level being mapped. 

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 
2001 U. S. History Assessment. 
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NAEP y.S. History Scale 




XfiA- 

jUU 



Advanced 






(itagfBBWaffisaBiB 

QL & ftteteo© 8@sfl© 
topai)©© 

Ifeo^feste 



Mmm (or 



acB@»iita 
QL&GMeog 






MM&ufeitiai 
HrpliKOMte sO 



spseiim, 0 



Proficient 



Basic 



332 Use pie charts to identify and interpret changes in colonial population 
-330 Recognize which of a group of sources is secondary 



322 Identify African American leaders 

319“ Identify political factor involved in adoption of Constitution 

317 Use map to identify and explain regional economic distinctions in particular era 

312 Interpret and put in historical context a Revolutionary-era image 



308 Interpret message of political cartoon about attitudes toward immigrants 

305 Identify impetus for a program of government-sponsored reform 

302 Explain why steel plow was important in U.S. history-— Sample Question 7 



295 Recognize Mesoam erica n group powerful at time of Columbus 



288 Describe effects on West of the expansion of the railroad system 
283 Interpret major Civil War-era speech 




272 Identify reason for Mormon migration to Utah 

269 Explain a cause of Civil War, with reference to northern and southern views 

261 Recognize and explain importance of particular technological changes 

258 Interpret a quotation from an immigrant to the United States — Sample Question 8 

256 Identify union strategy against factory owners 

248 Explain one way in which the railroads affected particular U.S. region 
246 Interpret the meaning of an important Civil War-era speech 
243 Categorize group of events as belonging to civil rights movement 

238 Identify importance of major event in civil rights movement 




227 Use photograph to infer purpose of late-1 9th-century photographer 



216 Identify major U.S . political leader of Revolutionary and early national periods 



u 

NOTE: Regular type denotes a constructed-response question. Italic type denotes a multiple-choice question. 

* Each grade 8 U. $. history question in the 2001 assessment was mapped onto the NAEP 0-500 U. $. history scale. The position of the question on the scale represents the scale score 
attained by students who had a 65 percent probability of successfully answering a constructed-response question, or a 74 percent probability of correctly answering a four-option 
multiple-choice question. Only selected questions are presented. Scale score ranges for U. S. history achievement levels are referenced on the map. For constructed-response questions, the 
question description represents students’ performance at the scoring criteria level being mapped. 

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 
2001 U. S. History Assessment. 
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Advanced 



366 Explain basic features of Constitutional three-fifths compromise — Sample Question 11 
365 Recognize significance of Gulf of Tonkin Resolution 



31 



Proficient 



Basic 



358 List a factor that led to US involvement in late-20th-century war 
,356, Identify major. point of Tjryan, " Qross.of, Gplci " sppeph, 



348 Interpret a political cartoon about 1960's foreign policy 
345 Describe conditions leading to 20th-century reform movement 
34 1 Identify conditions experienced by U.S. soldiers in World War I 



334 Describe two ways war affected society, use supporting evidence 

332 Recognize Bay of Pigs invasion of Cuba as subject of political cartoon 

330 Use cartoon to identify a slogan of Theodore Roosevelt's foreign policy 

328 Describe and explain religious institution's involvement in post-World War II reform 

3^4 identify changes in Cherokee* socfety and identify their consequent ’forced removal 



316 Identify book by Upton Sinclair about conditions in meat-packing industry 
313 Analyze passenger lists to infer contrasts between southern and New England colonies 
311 Explain differences between White and American Indian attitudes toward land ownership 
310 -Identify major cause of shift from indentured servant to slave labor in Virginia 

307 Infer from 1950’s job survey changing social attitudes toward women 
306 Recognize key assumption of 20th-century government reform program 
304 Use photograph to identify theme of 20th-century women's protest 



295 Recognize achievements of the Harlem Renaissance — Sample Question 10 



284 Identify a cause of 19th-century urban population explosion 

279 Identify meaning of 1787 quotation about superiority of small republics 
275 Use photograph to identify dates of 20th-century demographic shift 
272 Use photograph to identify a major event marking end of Cold War 






249 ~ Identify an effect of post-Civil 



War constitutional amendment 



H 

© 



NOTE: Regular type denotes a constructed-response question. Italic type denotes a multiple-choice question. 

* Each grade 12 U. S. history question in the 2001 assessment was mapped onto the NAEP 0-500 U. S. history scale. The position of the question on the scale represents the scale score 
attained by students who had a 65 percent probability of successfully answering a constructed-response question, or a 74 percent probability of correctly answering a four-option 
multiple-choice question. Only selected questions are presented. Scale score ranges for U. S. history achievement levels are referenced on the map. For constructed-response questions, the 
question description represents students’ performance at the scoring criteria level being mapped. 



SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 
2001 U. S. History Assessment. 
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Classroom Contexts for Learning 



This chapter presents information about practices that take 
place in and around the school that may affect the teaching 
and learning of U.S. history — the amount of time students 
spend in social studies classes, adherence to state and local 
standards for social studies education, classroom practices, 
and the use of technology. The information in this chapter 
is based on teachers’ and students’ responses to background 
questions administered as part of the NAEP 2001 
U.S. history assessment. The percentage of students 
and average scale scores are presented for each 
contextual variable reported in order to examine the 
relationship between students’ home and school 
experiences and their performance on the assessment. 
In interpreting these data, readers are reminded that 
the relationship between contextual variables and 
student performance is not necessarily causal. There 
are many factors that may play a role in a student’s 
performance on NAEP. 

Time Spent on Social Studies 

The NAEP U.S. History Framework that served 
as the blueprint for the 1994 and 2001 assessments 
recognized that most fourth-graders do not have a formal 
class in U.S. history. (Attention was paid in the framework to 
ensuring coverage of material that is likely to be addressed in 
fourth-grade classrooms, such as state history.) The majority 
of fourth-grade teachers, however, did report instructing 
their students regularly in social studies. More than one-half 
of fourth-graders had teachers who reported spending 




How much time 
do fourth-grade 
teachers devote 
to social studies 
instruction? 

How do classroom 
activities and 
computer use 
relate to student 
achievement? 
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between 61 and 180 minutes per week on 
social studies instruction in 2001. About 
one-fifth of the students had teachers who 
reported spending more than 180 minutes 
per week on social studies, and 14 percent 
had teachers who reported spending 60 
minutes or less. 

The results presented in table 6.1 show a 
generally positive relationship between the 
amount of time teachers reported spending 



on social studies instruction and fourth- 
graders’ performance. Students whose 
teachers reported spending 60 minutes or 
less per week had lower scores than those 
whose teachers reported spending more 
than 60 minutes per week. Fourth-graders 
whose teachers spent more than 180 
minutes per week on social studies had the 
highest average scores. 



Table S.1 

Percentage of students and average 
U.S. history scale scores by teachers’ reports 
on the amount of time spent on social studies 
in a typical week at grade 4: 2001 


Grade r 




sll gggp 


Less than 30 minutes 


2 

191 




30 to 60 minutes 


12 






195 




61 to 120 minutes 


37 






210 




121 to 180 minutes 


31 y 


^t)(DI)ai&Qooa^©a3@@C3 




211 / 




More than 180 minutes 


JY 






m 





The percentage of students is listed first with the corresponding average scale score presented below. 

NOTE: Percentages may not add to 100 due to rounding. 

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 2001 U.S. History Assessment. 
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State and Local Standards 

With the rise of the movement for stan- 
dards-based reform in K-12 education, an 
increasing number of states have adopted 
standards for history or social studies. 1 

In 2001, teachers of fourth- and eighth- 
grade students were asked about the extent 
to which they used state or local standards 
in planning their history or social studies 
instruction. Table 6.2 presents the percent- 
age of students and their average scores 
based on teachers’ responses to this ques- 
tion. Only a small percentage of students 
had teachers who reported that there were 
no state or local standards that applied to 
teaching social studies (3 percent at grade 



4, and 1 percent at grade 8). About two- 
thirds of the students had teachers who 
reported that standards were used to a large 
extent in planning instruction (63 percent 
at grade 4, and 69 percent at grade 8). 

There were, however, no statistically signifi- 
cant differences in students’ performance at 
either grade 4 or grade 8, based on 
whether or not there were standards or on 
the extent to which teachers reported 
using standards for planning social studies 
instruction. Because state and local stan- 
dards are diverse and are used in various 
ways, readers should interpret this data with 
caution. 



Council of Chief State School Officers (2000). Key state education pdicics on K-12 education (table 1 3, p. 23). 
Washington DC: Author 
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Table 6.2 

Percentage of students and 
average U.S. history scale scores 
by teachers’ reports on the use of 
stafe/local standards in planning 
instruction at grades 4 and 8: 2001 



Grade 4 

Not at all 



Grades 



No standards for teaching social studies 




mofState/Locall 

Standards insist* 

^ ? ■■■• jri 

Planning Instruction. 

1*11 1111 



2 

212 



Small extent 


9 




210 


Moderate extent 


23 




206 


Large extent 


2To\1 


No standards for teaching social studies 


3 




224 


Grade 8 




Not at all 


2 




274 


Small extent 


7 




264 


Moderate extent 


21 A 




my 


Large extent 


© 




m 






Q3©@0DaEf© raQi® asaa 

cffemasttesOsi^ 
@3i3iiO to [jCsbidOddeb 
MMEteat 



l 

276 



The percentage of students is listed first with the corresponding average scale score presented below. 

NOTE: Percentages may not add to 100 due to rounding. 

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 2001 U.S. History Assessment. 



Instructional Activities 

Fourth- and eighth-grade teachers whose 
students participated in the NAEP U.S. 
history assessments in 1994 and 2001 were 
asked a series of questions about the fre- 
quency with which they engaged their 
classes in certain types of instructional 
activities when teaching social studies or 
U.S. history. The results, presented in table 
6.3, vary somewhat by grade level and 
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show that most students had teachers who 
reported having them read from a textbook 
at least once or twice a week (88 percent at 
grade 4, and 90 percent at grade 8). In 
2001 , fourth-graders whose teachers asked 
them to read from a textbook on a daily 
basis had higher average scores than stu- 
dents whose teachers did so on a weekly or 
monthly basis. Although only a small 
percentage of fourth-graders had teachers 




who reported never or hardly ever asking 
them to read from a textbook, the average 
scores for these students were not signifi- 
cantly different from those whose teachers 
did so more frequently. In contrast, eighth- 
graders whose teachers reported never or 
hardly ever having them read from a 
textbook had higher scores than their peers 
whose teachers engaged in this activity 
daily, weekly, or monthly. Here again, this 
involved a small percentage of students. 

A large majority of fourth- and eighth- 
graders were in classes where teachers 
reported asking students to read extra 
materials that were not in the regular 
textbook, such as biographies or historical 
fiction, on at least an occasional basis 
(weekly or monthly). There were, however, 
no differences in students' performance at 
either grade, related to the frequency with 
which teachers reported having them read 
these extra materials. 

Teachers were also asked about the 
frequency of their use of primary historical 
documents, letters, diaries, or essays written 
by historical figures. While the use of such 
texts as part of history or social studies 
instruction was not related to student 
performance at grade 4, there was a 
positive relationship associated with 
weekly use of these materials at grade 8. 
Eighth-graders whose teachers reported 
using primary sources on a weekly basis 
had higher average scores than those 
whose teachers reported doing so monthly 
or never. 



A question that asked teachers about the 
frequency with which they engaged their 
students in writing reports revealed differ- 
ent performance patterns at grades 4 and 8. 
The average scores of fourth-graders whose 
teachers asked them to write reports 
weekly were lower than the scores of 
students whose teachers did so less fre- 
quently. There was no relationship be- 
tween the performance of eighth-graders 
and the frequency of writing reports. 

On the whole there has been little 
change between 1994 and 2001 in the 
percentage of students whose teachers 
reported various amounts of time spent in 
particular instructional activities. There 
were some exceptions to this pattern of 
stability, however. At grade 4, the percent- 
age of students with teachers who reported 
reading extra material once or twice a 
week increased from 33 percent in 1994 to 
44 percent in 2001, while the percentage 
with teachers reporting doing so once or 
twice a month decreased. The percentage 
of fourth-graders with teachers who used 
historical documents once or twice a month 
also increased, while the percentage with 
teachers who never or hardly ever used 
them decreased. At grade 8, the percentage 
of students whose teachers reported having 
them read extra material every day increased 
from 3 percent in 1994 to 7 percent in 
2001. The percentage of eighth-graders 
whose teachers never used primary histori- 
cal documents decreased from 1994 to 2001. 
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Table 6.3 

Percentage of students and average 
U.S. history scale scores by teachers’ reports 
on frequency of classroom activities at 
grades 4 and 8: 1994 and 2001 




Reading material from a textbook 
Almost every day 



Once or twice a week 



Once or twice a month 



Never or hardly ever 



Reading extra material not in the regular textbook 
Almost every day 



Once or twice a week 



Once or twice a month 



Never or hardly ever 



Using primary historical documents 
Almost every day 



Once or twice a week 



Once or twice a month 



Never or hardly ever 



Writing a report 
Almost every day 



Once or twice a week 



Once or twice a month 



Never or hardly ever 



43 

207 



44 

204 



8 

204 



5 

204 



6 

208 



33 

205 



46 

204 



15 

208 



1 

*** 



8 

201 



29 

208 



62 

205 



*** 

*** 



6 

188 



63 

207 



31 

205 



( 214 ’ 





[20 



5 
209 



9 

210 



44 ' 
211 



35 ’ 
208 



12 

208 



1 

*** 



11 

207 



39 ’ 
212 



48 ’ 

208 



1 

194 



6 

198 



59 

210 



34 

210 



ttUiJgUMI 

Gafj$D^§(B(^ 



See footnotes at end of table. > 



ERIC 



94 



CHAPTER 6 



U.S. HISTORY REPORT CARO 



107 



Table 6.3 (continued) 

Percentage of students and average 
U.S. history scale scores by teachers’ reports 
on frequency off classroom activities at 
grades 4 and 8: 1994 and 2001 



Grade 












Reading material from a textbook 

Almost every day 45 45 

259 264 


Once or twice a week 42 45 

259 262 


Once or twice a month 8 7 

266 262 


Never or hardly ever 5 3 

265 275 


Reading extra material not in the regular textbook 
Almost every day 3 7* * 

254 265 


Once or twice a week 32 37 

258 261 


Once or twice a month 47 44 

263 264 


Never or hardly ever 17 13 

258 264 


Using primary historical documents 

Almost every day 2 4 

268 264 


Once or twice a week 20 27 s’* 

260 (267)T 


Once or twice a month 55 54 

261 262 


Never or hardly ever 23 16* 

258 259 


Writing a report 

Almost every day # 1 

*** 255 


Once or twice a week 4 7 

256 266 


Once or twice a month 66 66 

261 263 




Never or hardly ever 



30 

259 



27 

263 



titoea^ 

G®l0aFM!l5®6(l^ 



ERIC 



The percentage of students is listed first with the corresponding average scale score presented below. 

*Significantly different from 1994. 

*** Sample size is insufficient to permit a reliable estimate. 

# Percentage is between 0.0 and 0.5. 

O :: Percentages may not add to 100 due to rounding. 

RCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of 
:ational Progress (NAEP), 1994 and 2001 U.S. History Assessments. 
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Table 6.4 presents the data for questions 
asked of twelfth-grade students that were 
similar to those asked of fourth- and 
eighth-grade teachers. A large majority of 
students reported reading from a textbook 
on a daily or weekly basis and had higher 
average scores than those who did so only 
a few times a year or never. Students who 
reported that they never read extra mate- 
rial, such as biographies or historical stories, 
not in the regular textbook had lower 
average scores than students who did so a 
few times a year or more. However, there 
was no statistically significant difference in 
the performance of students who read such 
extra material as frequently as every day 
compared to others who used it to a more 
limited extent, including those who did 
so as infrequently as a few times a year. 
Both twelfth-graders who reported never 
using letters, diaries, or essays written by 



historical people and those who reported 
doing so on a daily basis had lower average 
scores than students who reported engag- 
ing in this activity on a more moderate 
basis — weekly, monthly, or yearly. A similar 
pattern was evident in the results presented 
for writing reports. 

A general pattern noticeable between 
1994 and 2001 was that an increasing 
percentage of twelfth-graders reported 
regularly engaging in the instructional 
activities that they were asked about. The 
percentage of students who reported daily 
reading from a textbook increased from 
40 percent in 1994 to 44 percent in 2001. 
There were also higher percentages of 
twelfth-graders who reported using histori- 
cal documentation and writing reports 
daily, weekly, and monthly in 2001 than 
in 1994. 
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Table 6.4 

Percentage of students and average 
U.S. history scale scores by students’ 
reports on frequency of classroom 
activities at grade 12: 1994 and 2001 



Grade ^ 







Read material from a textbook 



About every day 


40 

289 


44* 

290 


Once or twice a week 


40 


38 




289 


289 


Once or twice a month 


9 


8 




284 


283 


A few times a year 


6 


6 




278 


276 


Never 


6 


5* 




268 


270 


Read extra material not in the regular textbook 


About every day 


9 


10 




288 


290 


Once or twice a week 


30 


31 




289 


291 


Once or twice a month 


24 


25 




291 


290 


A few times a year 


18 


17 




288 


289 / 


Never 


18 


A/ 




274 


_ &L . 



OlffiQSG, 



See footnotes at end of table. > 
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Table 6.4 (continued) 

Percentage of students and average 
U.S. history scale scores by students’ 

reports on frequency of classroom Grade \ 

activities at grade 12: 1994 and 2001 “ )) 


1 


Use letters, diaries, or essays written by historical people 

About every day 4 5 * 

280 280 


IMHMflsas© 

®0QMnfeD 

Oafgtoff 9®am 


Once or twice a week 14 18* 




Once or twice a month 24 26 * 

291 ^9iy- — 


A few times a year 26 26 / 

291 (292) 


Never 32 25 * 

279 279 


Write a report 

About every day 2 3 * 

267 271 


Once or twice a week 9 14* 

279 288 


Once or twice a month 35 41 * 

287 290 


A few times a year 40 34 * 

293 290 


Never 14 8 * 

278 271 


The percentage of students is listed first with the corresponding average scale score presented below. 
*$ignificantly different from 1994. 

NOTE: Percentages may not add to 100 due to rounding. 



SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 1994 and 2001 U.S. History Assessments. 
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Use of Technology 

Computer use in history and social studies 
education is an area of rapidly growing 
interest on the part of history educators. 2 
Fourth-, eighth-, and twelfth-grade students 
who participated in the 2001 assessment 
were asked a series of questions related to 
their use of computers for history and 
social studies both at school and at home. 
The results presented in tables 6.5 and 6.6 
show the relationship between students’ 
performance and their responses to three 
questions about their use of computers. 

Relatively few students reported using a 
computer for social studies or history. Only 
about one-quarter of the fourth-grade 
students reported using computers at 
school for social studies at least once every 
few weeks. This proportion increased to 
about one-third of the students at grade 8, 
and remained at about one-third among 
students at grade 12. 

The data presented in table 6.5 for grade 
4 and table 6.6 for grades 8 and 12, show 
that the reported frequency related to 
general usage of computers at school for 



history or social studies appeared to have a 
negative association with students’ perfor- 
mance at all three grades. Students in 
grades 4, 8, and 12 who reported daily use 
of computers at school for social studies all 
had lower average scores than those who 
reported less frequent use. There was not, 
however, a consistently negative association 
between computer use and students’ 
performance. A positive association with 
performance was evident for using com- 
puters specifically for conducting research 
and for writing reports at grades 8 and 12. 
At grades 8 and 12, students who used a 
CD or the Internet for research projects or 
used the computer to write reports more 
frequently had higher average scores than 
their peers who did so less frequently. This 
pattern did not hold true at grade 4, 
however, where there was no significant 
difference between the performance of 
those students who reported using a CD or 
the Internet for research projects or to 
write reports and those who reported that 
they did not. 



2 Martorella, P. H. (Ed.). (1997). Interactive technologies and the social studies : Emerging issues and applications. Albany, NY: 
State University of New York Press. 
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Table 6.5 

Percentage of students and average 

U.S. history scale scores by students’ reports 

on computer use at grade 4: 2001 



Use computers at school for social studies 

Every day 




Two or three times a week 


5 

186 


Once a week 


7 




197 


Once every few weeks 


10 




212 


Never or hardly ever 


74 




214 


Do research projects using a CD or the Internet 




Yes 


46 




211 


No 


54 




208 


Use computer to write reports 




Yes 


52 




209 



No 



48 

210 






G®©a89®OdED®Ora©8 

OtDmraBtnm 



The percentage of students is listed first with the corresponding average scale score presented below. 

NOTE: Percentages may not add to 100 due to rounding. 

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 2001 U.S. History Assessment. 
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Table 6.6 

Percentage of students and average 

U.S. history scale scores by students’ reports 

on computer use at grades 8 and 12: 2001 




Use computers at school for social studies 
Everyday 

Two or three times a week 



Once a week 



Once every few weeks 
Never or hardly ever 



Do research projects using a CD or the Internet 
Not at all 



Small extent 



Moderate extent 



Large extent 



Write reports on the computer 
Not at all 



Small extent 



Moderate extent 



Large extent 



1 

239 

5 

252 

9 

261 

21 

268 

64 

263 



26 

253 

31 

262 

26 

267 




OD^oaraeaim 



18 

253 

30 

260 

29 

266 

23 

270 



See footnotes at end of table. > 
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The percentage of students is listed first with the corresponding average scale score presented below. 

NOTE: Percentages may not add to 100 due to rounding. 

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 2001 U.S. History Assessment. 
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Appendix A 

OweiYiew of Procedures Used for the 
WAEP 2001 U. S. History Assessment 



This appendix provides an overview of the NAEP 2001 
history assessments primary components — framework, 
development, administration, scoring, and analysis. A more 
extensive review of the procedures and methods used in the 
history assessment will be included in the forthcoming 
NAEP 2001 Technical Report. 

The INAEP 2001 History Assessment 

The National Assessment Governing Board (NAGB), 
created by Congress in 1988, is responsible for 
formulating policy for NAEP. The NAGB is 
specifically charged with developing assessment 
objectives and test specifications through a national 
consensus approach. That consensus approach results 
in the development of an assessment framework. 

The design of the NAEP 2001 U.S. history 
assessment followed the guidelines provided in the 
framework developed for the 1994 assessment. 1 

The framework underlying both the NAEP 
1994 and 2001 assessments reflects current consensus 
among educators and researchers about the study of 
U.S. history. Developing this framework and the 
specifications that guided development of the assessment 
involved the critical input of hundreds of individuals across 
the country, including representatives of national education 





The Assessment 

The Samples 

Students with 
Disabilities (SO) 
and 

Limited English 
Proficient (LEP) 
Students 

Data Collection 

Data Analysis 

NAEP Reporting 
Groups 

Cautions in 
Interpretations 

& J 



1 National Assessment Governing Board (1994). U.S. history framework for the 1994 National 
Assessment of Educational Progress. Washington, DC: Author. 
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organizations, teachers, parents, policy- 
makers, business leaders, and the interested 
general public. This consensus process was 
managed by the Council of Chief State 
School Officers for NAGB. 

The assessment framework specified not 
only the particular aspects of U.S. history 
to be measured (see chapter 1 for a 
description of these aspects), but also the 
percentage of assessment questions that 
should be devoted to each. The target 
percentage distributions of historical 
themes, as specified in the framework, 
along with the actual percentage distribu- 
tions in the 1994 and 2001 assessments, are 
presented in table A.l. Notice that these 
percentages shift from grade 4 to grade 12 
to reflect the shift in curricular emphasis as 
students move from the fourth to the 
twelfth grade. For example, the emphasis 
on “the changing role of America in the 
world” grows at each successive grade level. 



It should also be noted that the actual 
content of the assessment has varied some- 
what in both 1994 and 2001 from the 
targeted distribution. At grades 8 and 12, 
these variances are in part explained by the 
use of “theme blocks,” which focus on 
particular historical themes and allow 
students to respond to a range of primary 
sources more extensively than would be 
possible in blocks that include the full 
spectrum of themes. In addition, it is 
evident from table A.l that some variance 
exists between the actual distribution of 
questions among the themes in 1994 and 
the actual distribution in 2001. These 
variances exist because seven new blocks 
were introduced in 2001 to replace blocks 
that were released to the public. Within the 
new blocks, the distribution of items 
differed somewhat from the distribution 
within the blocks that they replaced. 
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Target and actual percentage distribution of questions by historical theme, grades 4, 8, and 12: 
1994 and 2001 




fioatte'O 


OmU©® 


@503© 


Historical Themes 


Target 


Actual 

1994 


Actual 

2001 


1 

j Target 


Actual 

1994 


Actual 

2001 


Target 


Actual 

1994 


Actual 

2001 


Change and Continuity in 
American Democracy: 
Ideas, Institutions, 
Practices, and 
Controversies 


25 


24 


| 25 


j 

| 30 


28 


30 


25 


29 


28 


The Gathering and 
Interactions of Peoples, 
Cultures, and Ideas 


35 


32 


32 


1 

30 


30 


32 

j 


25 

| 


23 


26 


Economic and 
Technological Changes 
and Their Relation to 
Society, Ideas, and the 
Environment 


25 


25 


32 


> 

I 

20 


23 


25 


25 


26 


22 


The Changing Role of 
America in the World 


15 


19 


12 


20 


19 


13 


25 


21 


25 



SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP>, 1994 and 2001 U. $. History Assessments. 



The Assessment Design 

Each student who participated in the U.S. 
history assessment received a booklet 
containing three or four sections: a set of 
general background questions, a set of 
subject-specific background questions 
dealing largely with the student s use of 
technology, and one or two sets, or “blocks,” 
of cognitive questions assessing knowledge 
and skills in U.S. history as outlined in the 
framework. At grade 4, only 25-minute 
blocks were used. At grades 8 and 12, 
students were given either two 25-minute 
blocks or one 50-minute block. The 50- 
minute blocks administered at grades 8 and 
12 (one at each grade) included extended 
constructed-response questions requiring 



students to synthesize elements from 
various primary sources. 

At grade 4 a total of six blocks of cogni- 
tive questions were given, while at grades 8 
and 12 nine blocks were administered. 2 
Some of the blocks at each grade level 
(three at grade 4, six at grade 8, and six at 
grade 12) were carried forward from the 
1994 assessment to the 2001 assessment to 
allow for the measurement of change across 
time. Each block consisted of both mul- 
tiple-choice and constructed-response 
questions. Short constructed-response 
questions required a few sentences for an 
answer, while extended constructed- 
response questions generally required a 
paragraph or more. It was expected that 



2 These blocks were distributed across the student booklets in a Balanced Incomplete Block (BIB) design that is 
described later in this section. 
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students could adequately answer the short 
constructed-response questions in about 
two to three minutes and the extended 
constructed-response questions in about 
five minutes. 

The data in table A. 2 display the number 
of questions by type and by grade level for 
the 1994 and 2001 assessments. Some of 
these questions were used at more than one 
grade level; thus, the sum of the questions 
that appear at each grade level is greater 
than the total number of unique questions. 
The total number of questions at each 
grade level was slightly smaller in 2001 
than in 1994 because in 2001 there were 



slightly fewer extended constructed- 
response questions at grades 4 and 12, and 
fewer multiple-choice questions at grade 8. 
This decrease in the total number of 
questions simply reflects the fact that the 
new blocks that replaced blocks released to 
the public from the 1994 assessment 
contained slightly fewer questions. It 
should be noted that these variations across 
years do not affect the ability of NAEP to 
report changes in students’ performance 
across years since the estimated changes are 
based on the presence of blocks that were 
common to both assessment years. 




Distribution of questions administered by question type, grades 4, 8, and 12: 1994 and 2001 





(Boasted 


tea?® 


QoadtetJg 




1994 


2001 


1994 


2001 


1994 


2001 


Multiple-choice 


63 


63 


102 


99 ! 


103 


106 


Short constructed- 
response 


26 


28 


37 


39 


33 


35 


Extended constructed- 
response 


6 


3 


12 


7 


19 


i 

13 ! 


Total 


95 


94 


151 


145 


155 


154 



SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 1994 and 2001 U. S. History Assessments. 
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The assessment design allowed for 
maximum coverage of U.S. history content 
at grades 4 , 8, and 12, while minimizing 
the time burden for any one student. This 
was accomplished through the use of 
matrix sampling of questions, in which 
representative samples of students took 
various portions of the entire pool of 
assessment questions. The aggregate results 
across the entire assessment allowed for 
broad reporting of the U.S. history abilities 
for the targeted population. 

In addition to matrix sampling, the 
assessment design utilized a procedure for 
distributing booklets that controlled for 
position and context effects. Students 
received different blocks of questions in 
their booklets according to a procedure 
called “Balanced Incomplete Block (BIB) 
spiraling.” This procedure assigns blocks of 
questions so that every block appears in the 
first or second position within a booklet an 
equal number of times. Every block of 
questions is paired with every other block, 
with the exception of the 50-minute 
theme block, which appears on its own 
without another block of cognitive ques- 
tions. The spiraling aspect of this proce- 
dure cycles the booklets for administration, 
so that typically only a few students in any 
assessment session receive the same booklet. 
This design allows for some balancing of 
the impact of context and fatigue effects to 
be measured and reported, but makes 
allowance for the difficulties of administer- 
ing the 50-minute blocks. 3 



In addition to the student assessment 
booklets, three other instruments provided 
data relating to the assessment: a teacher 
questionnaire, a school questionnaire, and a 
Students with Disabilities/Limited-English 
Proficiency (SD and/or LEP) question- 
naire. The teacher questionnaire was 
administered to the history or social studies 
teachers of fourth- and eighth-grade 
students participating in the assessment. 

The questionnaire consisted of three 
sections and took approximately 20 min- 
utes to complete. The first section focused 
on the teacher’s general background and 
experience; the second section on com- 
puter resources available in the school; and 
the third section on classroom information 
about social studies instruction. 

The school characteristics and policy 
questionnaire was given to the principal or 
other administrator in each participating 
school and took about 20 minutes to 
complete. The questions asked about 
school policies, programs, facilities, and the 
demographic composition and background 
of the student body. 

The SD and/or LEP student question- 
naire was completed by a school staff mem- 
ber knowledgeable about those students 
who were selected to participate in the 
assessment and who were identified as: 

1) having an Individualized Education 
Program (IEP) or equivalent program 
(for reasons other than being gifted and 
talented) or 2) being limited English 



3 For further details on the booklet design, see the forthcoming NAEP 2001 Technical Report . 
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proficient (LEP). A questionnaire was 
completed for each SD and/or LEP stu- 
dent sampled regardless of whether the 
student participated in the assessment. Each 
questionnaire took approximately 3 min- 
utes to complete and asked about the 
student and the special programs in which 
he or she participated. 

National Sample 

The national results presented in this report 
are based on a nationally representative 
probability sample of fourth-, eighth-, and 
twelfth-grade students. The sample was 
chosen using a multistage design that 
involved sampling students from selected 
schools within selected geographic areas 
across the country. The sample design had 
the following stages: 

1) selection of geographic areas (a county, 
group of counties, or metropolitan 
statistical area); 

2) selection of schools (public and 
nonpublic) within the selected areas; and 

3) selection of students within selected 
schools. 

Each selected school that participated in 
the assessment and each student assessed 
represents a portion of the population of 
interest. Sampling weights are needed to 
make valid inferences between the student 
samples and the respective populations 
from which they were drawn. Sampling 
weights account for disproportionate 
representation due to the oversampling of 
students who attend schools with high 
concentrations of Black and/or Hispanic 
students and students who attend nonpublic 



schools. Among other uses, sampling 
weights also account for lower sampling 
rates for very small schools and are used to 
adjust for school and student nonresponse. 4 

Unlike the 1994 national assessments 
special feature of the 2001 national assess- 
ment was the collection of data from 
samples of students where assessment 
accommodations for special-needs students 
were not permitted and from samples of 
students where accommodations for 
special-needs students were permitted. 
NAEP inclusion rules were applied, and 
accommodations were offered only when a 
student had an Individualized Education 
Program (IEP) because of a disability and/ 
or was identified as being a limited English 
proficient student (LEP); all other students 
were asked to participate in the assessment 
under standard conditions. 

Table A. 3 shows the number of students 
included in the national samples for the 
NAEP 1994 and 2001 history assessments 
at each grade level. For the 2001 assess- 
ment, the table includes the number of 
students in the sample where accommoda- 
tions were not permitted and the number 
of students in the sample where accommo- 
dations were permitted. The table shows 
that the same non-SD and/or LEP students 
were included in both samples in 2001; 
only the SD and/or LEP students differed 
between the two samples. The 1994 design 
differed somewhat in that the SD and/or 
LEP students were assessed in standard 
conditions and accommodations were not 
permitted. 



0 
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4 Additional details regarding the design and structure of the national and state samples will be included in the 
forthcoming NAEP 2001 Technical Report. In addition, the reader may consult the NAEP 2000 Technical Report lor a 
discussion of sampling procedures that are mostly common to all NAEP assessments. 
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National student sample size by type of results, grades 4, 8, and 12: 1994 and 2001 








1994 




2001 








Accommodations 
not permitted 
sample 


Accommodations 
not permitted 
sample 




Accommodations 

permitted 

sample 


Grade 4 


Non SD/LEP students assessed 


5,067 




6,446 






SD/LEP students assessed 
without accommodations 


432 


581 




504 




SD/LEP students assessed 
with accommodations 


NA 


NA 




359 




Total students assessed 


5,499 


7,027 




7,309 


Grade 8 


Non SD/LEP students assessed 


8,227 




10,321 






SD/LEP students assessed 
without accommodations 


540 


918 




863 




SD/LEP students assessed 
with accommodations 


NA 


NA 




569 




Total students assessed 


8,767 


11,239 




11,753 


Grade 12 


Non SD/LEP students assessed 


7,427 




10,658 






SD/LEP students assessed 
without accommodations 


391 


658 




566 




SD/LEP students assessed 
with accommodations 


NA 


NA 




253 




Total students assessed 


7,818 


11,316 




11,477 



SD = Students with Disabilities. 

LEP = Limited English Proficient students. 

NA = Not applicable. No accommodations were permitted in this sample. 

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 1994 and 2001 U. S. History Assessments. 
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Table A. 4 provides a summary of the 
national school and student participation 
rates for the U.S. history assessment samples 
where accommodations were not per- 
mitted and where accommodations were 
permitted. Participation rates are presented 
for public and nonpublic schools, individu- 
ally and combined. The first rate is the 
weighted percentage of schools participat- 
ing in the assessment before substitution of 
demographically similar schools. 5 This rate 
is based only on the number of schools that 
were initially selected for the assessment. 
The numerator of this rate is the sum of 
the number of students represented by each 
initially selected school that participated in 
the assessment. The denominator is the 
sum of the number of students represented 
by each of the initially selected schools that 
had eligible students enrolled. 

The second school participation rate is 
the weighted participation rate after substi- 
tution. The numerator of this rate is the 
sum of the number of students represented 
by each of the participating schools, 
whether originally selected or selected as a 
substitute for a school that chose not to 
participate. The denominator is the same 
as that for the weighted participation rate 
for the initial sample. Because of the 
common denominators, the weighted 



participation rate after substitution is at 
least as great as the weighted participation 
rate before substitution. 

Also presented in table A. 4 are weighted 
student participation rates. The numerator 
of this rate is the sum across all students 
assessed (in either an initial session or a 
makeup session) of the number of students 
that each represents. The denominator of 
this rate is the sum across all eligible 
sampled students in participating schools of 
the number of students that each repre- 
sents. The overall participation rates take 
into account the weighted percentage of 
school participation before or after substi- 
tution and the weighted percentage of 
student participation after makeup sessions. 

For the grade 12 national sample, where 
school and student response rates did not 
meet NCES standards, an extensive analysis 
was conducted that examined, among 
other factors, the potential for nonresponse 
bias at both the school and student level. 

No evidence of any significant potential for 
either school or student nonresponse bias 
was found. Results of these analyses, as well 
as nonresponse bias analyses for the grade 4 
and grade 8 national samples, will be 
included in the forthcoming NAEP 2001 
Technical Report. 



a 

ERIC 

no 



5 The initial base sampling weights were used in weighting the percentages of participating schools and students. An 
attempt was made to preselect (before field processes began) a maximum of two substitute schools for each sampled 
public school (one in-district and one out-of-district) and each sampled Catholic school, and one for each sampled 
nonpublic school (other than Catholic). To minimize bias, a substitute school resembled the original selection as 
much as possible on affiliation, estimated number of grade-eligible students, and minority composition. 
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National school and student participation rates for public schools, nonpublic schools, and public 
and nonpublic schools combined, grades 4, 8, and 12: 2001 





I 

j Weighted school participation 


Samples where 
were not 


accommodations 

permitted 


Samples where accommodations 
were permitted 










Student participation 


Overall participation rate 


Student participation 


Overall participation rate 




Percentage 

before 


Percentage 

after 


Total 

number 


Weighted 

percentage 

student 


Total 

number of 
students 


Before 


After 


Weighted 

percentage 

student 


Total 

number of 
students 


Before 


After 




substitution 


substitution 


of schools 


participation 


assessed 


substitution 


substitution 


participation 


assessed 


substitution 


substitution 


Grade 4 
























Public 


83 


88 


276 


96 


5,978 


80 


84 


96 


6,266 


80 


84 


Nonpublic 


83 


91 


89 


97 


1,049 


81 


88 


97 


1,043 


81 


88 


Combined 


83 


88 


365 


96 


7,027 


80 


85 


96 


7,309 


80 


85 


Grade 8 
























Public 


79 


87 


259 


93 


9,694 


73 


81 


93 


10,180 


73 


81 


Nonpublic 


84 


88 


110 


96 


1,561 


81 


84 


96 


1,582 


80 


84 


Combined 


79 


87 


369 


93 


11,255 


74 


81 


93 


11,762 


74 


81 


Grade 12 
























Public 


73 


80 


311 


77 


10,051 


56 


62 


76 


10,220 


56 


61 


Nonpublic 


67 


77 


63 


90 


1,265 


61 


70 


90 


1,257 


61 


70 


Combined 


72 


80 


374 


78 


11,316 


56 


62 


77 


11,477 


56 


62 



SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of 
Educational Progres (NAEP), 2001 U. S. History Assessment. 



Students with Disabilities (SD) 
and/or Limited English Proficient 
(LEP) Students 

It is NAEP’s intent to assess all selected 
students from the target population. 
Therefore, every effort is made to ensure 
that all selected students who are capable of 
participating in the assessment are assessed. 
Some students sampled for participation in 
NAEP can be excluded from the sample 
according to carefully defined criteria. 

These criteria were revised in 1996 to 
communicate more clearly a presumption 
of inclusion except under special circum- 
stances. According to these criteria, stu- 



dents with Individualized Education 
Programs (IEPs) were to be included in the 
NAEP assessment except in the following 
cases: 

1) The schools IEP team determined that 
the student could not participate, OR, 

2) The students cognitive functioning was 
so severely impaired that she or he could 
not participate, OR, 

3) The students IEP required that the student 
had to be tested with an accommodation 
or adaptation and that the student could 
not demonstrate his or her knowledge 
without that accommodation. 6 



6 As described in the following section, a second sample in the 2001 national assessments was assessed that included 
students who required and were provided with accommodations. 
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